sidekick.seq

Basic types

iter	Base sidekick iterator class.
generator	Decorates generator function to return a sidekick iterator instead of a regular Python generator.

Basic manipulation of sequences

cons	Construct operation.
uncons	De-construct sequence.
first	Return the first element of sequence.
second	Return the second element of sequence.
nth	Return the nth element in a sequence.
find	Return the (position, value) of first element in which predicate is true.
only	Return the single element of sequence or raise an error.
last	Return last item (or items) of sequence.
is_empty	Return True if sequence is empty.
length	Return length of sequence, consuming the iterator.
consume	Consume iterator for its side-effects and return last value or None.

Creating new sequences

cycle	Return elements from the iterable until it is exhausted.
iterate	Repeatedly apply a function func to input.
iterate_indexed	Similar to iterate(), but also pass the index of element to func.
repeat	for the specified number of times.
repeatedly	Make infinite calls to a function with the given arguments.
singleton	Return iterator with a single object.
unfold	Invert a fold.

Filtering and select sub-sequences

filter	Return an iterator yielding those items of iterable for which function(item) is true.
remove	Opposite of filter.
separate	Split sequence it two.
drop	Drop items from the start of iterable.
rdrop	Drop items from the end of iterable.
take	Return the first entries iterable.
rtake	Return the last entries iterable.
unique	Returns the given sequence with duplicates removed.
dedupe	Remove duplicates of successive elements.
converge	Test convergence with the predicate function by passing the last two items of sequence.

Grouping items

group_by	Group collection by the results of a key function.
chunks	Partition sequence into non-overlapping tuples of length n.
chunks_by	Partition sequence into chunks according to a function.
window	Return a sequence of overlapping sub-sequences of size n.
pairs	Returns an iterator of a pair adjacent items.
partition	Partition sequence in two.
distribute	Distribute items of seq into n different sequences.

Reducing sequences

fold	Perform a left reduction of sequence.
reduce	Like fold, but does not require initial value.
scan	Returns a sequence of the intermediate folds of seq by func.
acc	Like scan(), but uses first item of sequence as initial value.
fold_by	Reduce each sequence generated by a group by.
reduce_by	Similar to fold_by, but only works on non-empty sequences.
fold_together	Folds using multiple functions simultaneously.
reduce_together	Similar to fold_together, but only works on non-empty sequences.
scan_together	Folds using multiple functions simultaneously.
acc_together	Similar to fold_together, but only works on non-empty sequences.
product	Multiply all elements of sequence.
products	Return a sequence of partial products.
sum	Sum all arguments of sequence.
sums	Return a sequence of partial sums.
all_by	Return True if all elements of seq satisfy predicate.
any_by	Return True if any elements of seq satisfy predicate.
top_k	Find the k largest elements of a sequence.

API reference

class sidekick.seq.iter

Base sidekick iterator class.

This class extends classical Python iterators with a few extra operators. Sidekick iterators accepts slicing, indexing, concatenation (with the + sign) repetition (with the * sign) and pretty printing.

Operations that return new iterators (e.g., slicing, concatenation, etc) consume the data stream. Operations that simply peek at data execute the generator (and thus may produce side-effects), but cache values and do not consume data stream.

copy() → sidekick.seq.iter.iter

Return a copy of iterator. Consuming the copy do not consume the original iterator.

Internally, this method uses itertools.tee to perform the copy. If you known that the iterator will be consumed, it is faster and more memory efficient to convert it to a list and produce multiple iterators.

peek(n: int) → Tuple

Peek the first n elements without consuming the iterator.

tee(n=1) → Tuple[sidekick.seq.iter.iter]

Split iterator into n additional copies.

The copy method is simply an alias to iter.tee(1)[0]

sidekick.seq.generator

Decorates generator function to return a sidekick iterator instead of a regular Python generator.

Examples

>>> @sk.generator
... def fibonacci():
...     x = y = 1
...     while True:
...         yield x
...         x, y = y, x + y
>>> fibonacci()
sk.iter([1, 1, 2, 3, 5, 8, ...])

sidekick.seq.cons

Construct operation. Add x to beginning of sequence.

Examples

>>> sk.cons(0, range(1, 6))
sk.iter([0, 1, 2, 3, 4, 5])

See also

uncons()

sidekick.seq.uncons

De-construct sequence. Return a pair of (first, *rest) of sequence.

If default is given and if seq is an empty sequence return (default, empty_sequence), otherwise raise a ValueError.

Examples

>>> head, tail = sk.uncons(range(6))
>>> head
0
>>> tail
sk.iter([1, 2, 3, 4, 5])

See also

cons() first()

sidekick.seq.first

Return the first element of sequence.

Raise ValueError or return the given default if sequence is empty.

Examples

>>> sk.first("abcd")
'a'

See also

second() last() nth()

sidekick.seq.second

Return the second element of sequence.

Raise ValueError or return the given default if sequence has last than 2 elements.

Examples

>>> sk.second("abcd")
'b'

See also

first() last() nth()

Notes

There is no third, fourth, etc, because we can easily create those functions using nth(n). Sidekick implements first/second to help selecting items of a pair, which tends to appear frequently when working with dictionaries.

sidekick.seq.nth

Return the nth element in a sequence.

Return the default if sequence is not large enough or raise a ValueError if default is not given.

Warning

If seq is an iterator, consume the first n items.

Examples

>>> sk.nth(2, "abcd")
'c'

See also

first() second() last()

sidekick.seq.find

Return the (position, value) of first element in which predicate is true.

Raise ValueError if sequence is exhausted without a match.

Examples

>>> sk.find((X == 11), [2, 3, 5, 7, 11, 13, 17])
(4, 11)

sidekick.seq.only

Return the single element of sequence or raise an error.

Parameters:

• seq – Input sequence.
• default – Optional default value, returned if sequence is empty.

Examples

>>> sk.only([42])
42
>>> sk.only([], default=42)
42
>>> sk.only([42, 43])
Traceback (most recent call last):
...
ValueError: sequence is too long

sidekick.seq.last

Return last item (or items) of sequence.

Parameters:

• seq – Input sequence
• default – If given, and sequence is empty, return it. An empty sequence with no default value raises a ValueError.
• n – If given, return a tuple with the last n elements instead. If default is given and sequence is not large enough, fill it with the value, otherwise raise a ValueError.

Examples

>>> sk.last("abcd")
'd'

Notes

If you don’t want to raise errors if sequence is not large enough, use rtake():

>>> tuple(sk.rtake(5, "abc"))  # No error!
('a', 'b', 'c')
>>> sk.last("abc", n=5, default="-")
('-', '-', 'a', 'b', 'c')

See also

first() second() nth() rtake()

sidekick.seq.is_empty

Return True if sequence is empty.

Warning

This function consume first element of iterator. Use this only to assert that some iterator was consumed without using it later or create a copy with itertools.tee that will preserve the consumed element.

Examples

>>> nums = iter(range(5))
>>> sum(nums)  # exhaust iterator
10
>>> sk.is_empty(nums)
True

sidekick.seq.length

Return length of sequence, consuming the iterator.

If limit is given, consume sequence up to the given limit. This is useful to test if sequence is longer than some given size but without consuming the whole iterator if so.

Examples

>>> sk.length(range(10))
10

sidekick.seq.consume

Consume iterator for its side-effects and return last value or None.

Parameters:

• seq – Any iterable
• default – Fallback value returned for empty sequences.

Examples

>>> it = map(print, [1, 2, 3])
>>> sk.consume(it)
1
2
3

sidekick.seq.cycle

Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely.

cycle(seq) ==> seq[0], seq[1], …, seq[n - 1], seq[0], seq[1], …

Examples

>>> sk.cycle([1, 2, 3])
sk.iter([1, 2, 3, 1, 2, 3, ...])

sidekick.seq.iterate

Repeatedly apply a function func to input.

If more than one argument to func is passed, it iterate over the past n values. It requires at least one argument, if you need to iterate a zero argument function, call repeatedly()

Iteration stops if if func() raise StopIteration.

iterate(f, x) ==> x, f(x), f(f(x)), …

Examples

Simple usage, with a single argument. Produces powers of two.

>>> sk.iterate((X * 2), 1)
sk.iter([1, 2, 4, 8, 16, 32, ...])

Now we call with two arguments to func to produce Fibonacci numbers

>>> sk.iterate((X + Y), 1, 1)
sk.iter([1, 1, 2, 3, 5, 8, ...])

See also

repeatedly()

sidekick.seq.iterate_indexed

Similar to iterate(), but also pass the index of element to func.

iterate_indexed(f, x) ==> x, f(0, x), f(1, <previous>), …

Parameters:

• func – Iteration function (index, value) -> next_value.
• x – Initial value of iteration.
• idx – Sequence of indexes. If not given, uses start, start + 1, …
• start – Starting value for sequence of indexes.

Examples

>>> sk.iterate_indexed(lambda i, x: i * x, 1, start=1)
sk.iter([1, 1, 2, 6, 24, 120, ...])

sidekick.seq.repeat

for the specified number of times. If not specified, returns the object endlessly.

Examples

>>> sk.repeat(42, times=5)
sk.iter([42, 42, 42, 42, 42])

sidekick.seq.repeatedly

Make infinite calls to a function with the given arguments.

Stop iteration if func() raises StopIteration.

Examples

>>> sk.repeatedly(list, (1, 2))
sk.iter([[1, 2], [1, 2], [1, 2], [1, 2], [1, 2], ...])

sidekick.seq.singleton

Return iterator with a single object.

Examples

>>> sk.singleton(42)
sk.iter([42])

sidekick.seq.unfold

Invert a fold.

Similar to iterate, but expects a function of seed -> (seed’, x). The second value of the tuple is included in the resulting sequence while the first is used to seed func in the next iteration. Stops iteration if func returns None or raise StopIteration.

Examples

>>> sk.unfold(lambda x: (x + 1, x), 0)
sk.iter([0, 1, 2, 3, 4, 5, ...])

sidekick.seq.filter

Return an iterator yielding those items of iterable for which function(item) is true.

Behaves similarly to Python’s builtin filter, but accepts anything convertible to callable using sidekick.functions.to_callable() as predicate and return sidekick iterators instead of regular ones.

filter(pred, seq) ==> seq[a], seq[b], seq[c], …

in which a, b, c, … are the indexes in which pred(seq[i]) == True.

Examples

>>> sk.filter((X % 2), range(10))
sk.iter([1, 3, 5, 7, 9])

See also

remove() separate()

sidekick.seq.remove

Opposite of filter. Return those items of sequence for which pred(item) is False

Examples

>>> sk.remove((X < 5), range(10))
sk.iter([5, 6, 7, 8, 9])

See also

filter(). separate()

sidekick.seq.separate

Split sequence it two. The first consists of items that pass the predicate and the second of those items that don’t.

Similar to (filter(pred, seq), filter(!pred, seq)), but only evaluate the predicate once per item.

Parameters:

• pred – Predicate function
• seq – Iterable of items that should be separated.
• consume – If given, fully consume the iterator and return two lists. This is faster than separating and then converting each element to a list.

Examples

>>> sk.separate((X % 2), [1, 2, 3, 4, 5])
(sk.iter([1, 3, 5]), sk.iter([2, 4]))

See also

filter() remove()

sidekick.seq.drop

Drop items from the start of iterable.

If key is a number, drop at most this number of items for iterator. If it is a predicate, drop items while key(item) is true.

drop(key, seq) ==> seq[n], seq[n + 1], …

n is either equal to key, if its a number or is the index for the first item in which key(item) is false.

Examples

>>> sk.drop((X < 5), range(10))
sk.iter([5, 6, 7, 8, 9])

>>> sk.drop(3, range(10))
sk.iter([3, 4, 5, 6, 7, 8, ...])

See also

take() rdrop()

sidekick.seq.rdrop

Drop items from the end of iterable.

Examples

>>> sk.rdrop(2, [1, 2, 3, 4])
sk.iter([1, 2])

>>> sk.rdrop((X >= 2), [1, 2, 4, 1, 2, 4])
sk.iter([1, 2, 4, 1])

See also

drop() rtake()

sidekick.seq.take

Return the first entries iterable.

If key is a number, return at most this number of items for iterator. If it is a predicate, return items while key(item) is true.

take(key, seq) ==> seq[0], seq[1], …, seq[n - 1]

n is either equal to key, if its a number or is the index for the first item in which key(item) is false.

This function is a complement of drop(). Given two identical iterators seq1 and seq2, take(key, seq1) + drop(key, seq2) yields the original sequence of items.

Examples

>>> sk.take((X < 5), range(10))
sk.iter([0, 1, 2, 3, 4])

See also

drop()

sidekick.seq.rtake

Return the last entries iterable.

Examples

>>> sk.rtake(2, [1, 2, 3, 4])
sk.iter([3, 4])

>>> sk.rtake((X >= 2), [1, 2, 4, 1, 2, 4])
sk.iter([2, 4])

See also

take() rdrop()

sidekick.seq.unique

Returns the given sequence with duplicates removed.

Preserves order. If key is supplied map distinguishes values by comparing their keys.

Parameters:

• seq – Iterable of objects.
• key – Optional key function. It will return only the first value that evaluate to a unique key by the key function.
• exclude – Optional sequence of keys to exclude from seq
• slow – If True, allows the slow path (i.e., store seen elements in a list, instead of a hash).

Examples

>>> sk.unique(range(10), key=(X % 5))
sk.iter([0, 1, 2, 3, 4])

Note

Elements of a sequence or their keys should be hashable, otherwise it must use a slow path.

See also

dedupe()

sidekick.seq.dedupe

Remove duplicates of successive elements.

Parameters:

• seq – Iterable of objects.
• key – Optional key function. It will yield successive values if their keys are different.

See also

unique()

sidekick.seq.converge

Test convergence with the predicate function by passing the last two items of sequence. If pred(penultimate, last) returns True, stop iteration.

Examples

We start with a converging (possibly infinite) sequence and an explicit criteria.

>>> seq = sk.iterate((X / 2), 1.0)
>>> conv = lambda x, y: abs(x - y) < 0.01

Run it until convergence

>>> it = sk.converge(conv, seq); it
sk.iter([1.0, 0.5, 0.25, 0.125, 0.0625, 0.03125, ...])
>>> sum(it)
1.9921875

sidekick.seq.fold

Perform a left reduction of sequence.

Examples

>>> sk.fold(op.add, 0, [1, 2, 3, 4])
10

See also

reduce() scan()

sidekick.seq.reduce

Like fold, but does not require initial value.

This function raises a ValueError on empty sequences.

Examples

>>> sk.reduce(op.add, [1, 2, 3, 4])
10

See also

fold() acc()

sidekick.seq.scan

Returns a sequence of the intermediate folds of seq by func.

In other words it generates a sequence like:

func(init, seq[0]), func(result[0], seq[1]), …

in which result[i] corresponds to items in the resulting sequence.

See also

acc() fold()

sidekick.seq.acc

Like scan(), but uses first item of sequence as initial value.

See also

scan() reduce()

sidekick.seq.reduce_by

Similar to fold_by, but only works on non-empty sequences.

Initial value is taken to be the first element in sequence.

Examples

>>> sk.reduce_by(X % 2, op.add, [1, 2, 3, 4, 5])
{1: 9, 0: 6}

See also

fold() group_by() reduce_by()

sidekick.seq.fold_by

Reduce each sequence generated by a group by.

More efficient than performing separate operations since it does not store intermediate groups.

Examples

>>> sk.fold_by(X % 2, op.add, 0, [1, 2, 3, 4, 5])
{1: 9, 0: 6}

See also

fold() group_by() reduce_by()

sidekick.seq.fold_together

Folds using multiple functions simultaneously.

Examples

>>> seq = [1, 2, 3, 4, 5]
>>> sk.fold_together(seq, sum=((X + Y), 0), prod=((X * Y), 1))
{'sum': 15, 'prod': 120}

sidekick.seq.reduce_together

Similar to fold_together, but only works on non-empty sequences.

Initial value is taken to be the first element in sequence.

Examples

>>> seq = [1, 2, 3, 4, 5]
>>> sk.reduce_together(seq, sum=(X + Y), prod=(X * Y), max=max, min=min)
{'sum': 15, 'prod': 120, 'max': 5, 'min': 1}

sidekick.seq.scan_together

Folds using multiple functions simultaneously.

Initial value is passed as a tuple for each (operator, value) keyword argument.

Examples

>>> seq = [1, 2, 3, 4, 5]
>>> for acc in sk.scan_together(seq, sum=(X + Y, 0), prod=(X * Y, 1)):
...     print(acc)
{'sum': 0, 'prod': 1}
{'sum': 1, 'prod': 1}
{'sum': 3, 'prod': 2}
{'sum': 6, 'prod': 6}
{'sum': 10, 'prod': 24}
{'sum': 15, 'prod': 120}

sidekick.seq.acc_together

Similar to fold_together, but only works on non-empty sequences.

Initial value is taken to be the first element in sequence.

Examples

>>> seq = [1, 2, 3, 4, 5]
>>> for acc in sk.acc_together(seq, sum=(X + Y), prod=(X * Y)):
...     print(acc)
{'sum': 1, 'prod': 1}
{'sum': 3, 'prod': 2}
{'sum': 6, 'prod': 6}
{'sum': 10, 'prod': 24}
{'sum': 15, 'prod': 120}

sidekick.seq.product

Multiply all elements of sequence.

Examples

>>> sk.product([1, 2, 3, 4, 5])
120

See also

products() sum()

sidekick.seq.products

Return a sequence of partial products.

Examples

>>> sk.products([1, 2, 3, 4, 5])
sk.iter([1, 2, 6, 24, 120])

See also

acc() sums()

sidekick.seq.sum

Sum all arguments of sequence.

It exists only in symmetry with product(), since Python already has a builtin sum function that behaves identically.

Examples

>>> sk.sum([1, 2, 3, 4, 5])
15

See also

sums() product()

sidekick.seq.sums

Return a sequence of partial sums.

Same as sk.fold((X + Y), seq, 0)

Examples

>>> sk.sums([1, 2, 3, 4, 5])
sk.iter([1, 3, 6, 10, 15])

See also

acc() products()

sidekick.seq.all_by

Return True if all elements of seq satisfy predicate.

all_by(None, seq) is the same as the builtin all(seq) function.

Examples

>>> sk.all_by((X % 2), [1, 3, 5])
True

See also

any_by()

sidekick.seq.any_by

Return True if any elements of seq satisfy predicate.

any_by(None, seq) is the same as the builtin any(seq) function.

Examples

>>> sk.any_by(sk.is_divisible_by(2), [2, 3, 5, 7, 11, 13])
True

See also

all_by()

sidekick.seq.top_k

Find the k largest elements of a sequence.

Examples

>>> sk.top_k(3, "hello world")
('w', 'r', 'o')

sidekick.seq.group_by

Group collection by the results of a key function.

Examples

>>> sk.group_by((X % 2), range(5))
{0: [0, 2, 4], 1: [1, 3]}

See also

reduce_by() fold_by()

sidekick.seq.chunks

Partition sequence into non-overlapping tuples of length n.

Parameters:

• n – Number of elements in each partition. If n is a sequence, it selects partitions by the sequence size.
• seq – Input sequence.
• pad – If given, pad a trailing incomplete partition with this value until it has n elements.
• drop – If True, drop the last partition if it has less than n elements.

Examples

Too see the difference between padding, dropping and the regular behavior.

>>> sk.chunks(2, range(5))
sk.iter([(0, 1), (2, 3), (4,)])

>>> sk.chunks(2, range(5), pad=None)
sk.iter([(0, 1), (2, 3), (4, None)])

>>> sk.chunks(2, range(5), drop=True)
sk.iter([(0, 1), (2, 3)])

Using sequences, we can create more complicated chunking patterns.

>>> sk.chunks(sk.cycle([2, 3]), range(10))
sk.iter([(0, 1), (2, 3, 4), (5, 6), (7, 8, 9)])

A trailing ellipsis consumes the rest of the iterator.

>>> sk.chunks([1, 2, 3, ...], range(10))
sk.iter([(0,), (1, 2), (3, 4, 5), (6, 7, 8, 9)])

See also

chunks_by() window()

sidekick.seq.chunks_by

Partition sequence into chunks according to a function.

It creates a new partition every time the value of func(item) changes.

Parameters:

• func – Function used to control partition creation
• seq – Input sequence.
• how –

  Control how func is used to create new chunks from iterator.

  • ’values’ (default): create a new chunk when func(x) changes value

  • ’pairs’: create new chunk when func(x, y) for two successive values

    is True.

  • ’left’: create new chunk when func(x) is True. x is put in the

    chunk to the left.

  • ’right’: create new chunk when func(x) is True. x is put in the chunk to the right.

  • ’drop’: create new chunk when func(x) is True. x dropped from output

    sequence. It behaves similarly to str.split.

Examples

Standard chunker

>>> sk.chunks_by((X // 3), range(10))
sk.iter([(0, 1, 2), (3, 4, 5), (6, 7, 8), (9,)])

Chunk by pairs

>>> sk.chunks_by((Y <= X), [1, 2, 3, 2, 4, 8, 0, 1], how='pairs')
sk.iter([(1, 2, 3), (2, 4, 8), (0, 1)])

Chunk by predicate. The different versions simply define in which chunk the split location will be allocated

>>> sk.chunks_by(sk.is_odd, [1, 2, 3, 2, 4, 8], how='left')
sk.iter([(1,), (2, 3), (2, 4, 8)])
>>> sk.chunks_by(sk.is_odd, [1, 2, 3, 2, 4, 8], how='right')
sk.iter([(1, 2), (3, 2, 4, 8)])
>>> sk.chunks_by(sk.is_odd, [1, 2, 3, 2, 4, 8], how='drop')
sk.iter([(), (2,), (2, 4, 8)])

See also

chunks() partition()

sidekick.seq.window

Return a sequence of overlapping sub-sequences of size n.

n == 2 is equivalent to a pairwise iteration.

Examples

Pairwise iteration:

>>> [''.join(p) for p in sk.window(2, "hello!")]
['he', 'el', 'll', 'lo', 'o!']

See also

pairs()

sidekick.seq.pairs

Returns an iterator of a pair adjacent items.

This is similar to window(2), but requires a fill value specified either with prev or next to select if it will form pairs with the preceeding of the following value.

It must specify either prev or next, never both.

Parameters:

• seq – Input sequence.
• prev – If given, fill this value in the first item and iterate over all items preceded with the previous value.
• next – If given, fill this value in the last item and iterate over all items followed with the next value.

Examples

>>> [''.join(p) for p in sk.pairs("hello!", prev="-")]
['-h', 'he', 'el', 'll', 'lo', 'o!']
>>> [''.join(p) for p in sk.pairs("hello!", next="!")]
['he', 'el', 'll', 'lo', 'o!', '!!']

See also

window()

sidekick.seq.partition

Partition sequence in two.

Returns a sequence with elements before and after the key separator.

Parameters:

• key – An integer index or predicate used to partition sequence.
• seq – Input sequence.

Examples

>>> a, b = sk.partition(2, [5, 4, 3, 2, 1])
>>> a
sk.iter([5, 4])
>>> b
sk.iter([3, 2, 1])

>>> a, b = sk.partition((X == 3), [1, 2, 3, 4, 5])
>>> a
sk.iter([1, 2])
>>> b
sk.iter([3, 4, 5])

sidekick.seq.distribute

Distribute items of seq into n different sequences.

Parameters:

• n – Number of output sequences.
• seq – Input sequences.

Examples

>>> a, b = sk.distribute(2, [0, 1, 2, 3, 4, 5, 6])
>>> a
sk.iter([0, 2, 4, 6])
>>> b
sk.iter([1, 3, 5])