architxt.bucket

architxt.bucket#

Classes

TreeBucket()

Abstract base class for a scalable, persistent, transactional container of Tree objects.
class architxt.bucket.TreeBucket[source]#

Bases: ABC, MutableSet, Collection

Abstract base class for a scalable, persistent, transactional container of Tree objects.

TreeBucket behaves like a set while providing durable storage and explicit transactional semantics. It is designed for large-scale data and supports millions of trees with bounded memory usage through batched commits.

Transaction Management

  • Adding or removing trees requires an active transaction.

  • Modifying a tree that is already in a bucket is not possible without a transaction. The modifications are automatically persisted when the transaction is committed.

  • Exceptions inside a with bucket.transaction(): block automatically roll back the transaction.

Available Implementations

digraph inheritanceb98aa18bd2 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ABC" [URL="https://docs.python.org/3/library/abc.html#abc.ABC",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Helper class that provides a standard way to create an ABC using"]; "Collection" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.Collection",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Sized" -> "Collection" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Iterable" -> "Collection" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Container" -> "Collection" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Container" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.Container",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Iterable" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "MutableSet" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.MutableSet",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A mutable set is a finite, iterable container."]; "Set" -> "MutableSet" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Set" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.Set",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A set is a finite, iterable container."]; "Collection" -> "Set" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Sized" [URL="https://docs.python.org/3/library/collections.abc.html#collections.abc.Sized",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "TreeBucket" [URL="#architxt.bucket.TreeBucket",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Abstract base class for a scalable, persistent, transactional container of :py:class:`~architxt.tree.Tree` objects."]; "ABC" -> "TreeBucket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MutableSet" -> "TreeBucket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Collection" -> "TreeBucket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "ZODBTreeBucket" [URL="architxt.bucket.zodb.html#architxt.bucket.zodb.ZODBTreeBucket",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A persistent, scalable container for :py:class:`~architxt.tree.Tree` objects backed by ZODB and RelStorage using SQLite."]; "TreeBucket" -> "ZODBTreeBucket" [arrowsize=0.5,style="setlinewidth(0.5)"]; }
async async_update(trees, *, commit=False)[source]#

Asynchronously add multiple Tree to the bucket.

This method mirrors the behavior of update() but supports asynchronous iteration.

Parameters:

  • trees (Union[Iterable[Tree], AsyncIterable[Tree]]) – Trees to add to the bucket.

  • commit (Union[bool, int]) – Commit automatically. If already in a transaction, no commit is applied. - If False (default), no commits are made, it relies on the current transaction. - If True, commits in batch. - If an integer, commits every N tree. To avoid memory issues, we recommend using incremental commit with large iterables.

Return type:

None

abstractmethod close()[source]#

Close the underlying storage and release any associated resources.

Return type:

None

abstractmethod get_persistent_ref(tree)[source]#

Get a persistent reference for a given tree.

Parameters:

tree (Tree) – The tree to get the persistent reference for.

Return type:

Any

Returns:

The persistent reference of the tree for this bucket.

Raises:

KeyError – If the tree is not stored in the bucket.

abstractmethod oids()[source]#

Yield the object IDs (OIDs) of all trees stored in the bucket.

Return type:

Generator[UUID, None, None]

abstractmethod resolve_ref(ref)[source]#

Resolve a persistent_ref back to a live Tree instance.

Parameters:

ref (Any) – The value returned by Tree.persistent_ref().

Return type:

Tree

Returns:

The tree corresponding to the given persistent reference.

Raises:

KeyError – If the tree is not found in the bucket.

abstractmethod sync()[source]#

Synchronize the in-memory state of this bucket with its underlying storage.

Implementations typically flush, refresh and/or invalidate caches and reload metadata so that subsequent operations reflect external changes. This operation may be expensive, so it should be called sparingly, but it is often required in concurrent environments (e.g., when using threads or subprocesses).

Return type:

None

abstractmethod transaction()[source]#

Return a context manager for managing a transaction.

Upon exiting the context, the transaction is automatically committed. If an exception occurs within the context, the transaction is rolled back.

Transactions are reentrant.

Return type:

AbstractContextManager[None]

update(trees, *, commit=False)[source]#

Add multiple Tree to the bucket.

Parameters:

  • trees (Iterable[Tree]) – Trees to add to the bucket.

  • commit (Union[bool, int]) – Commit automatically. If already in a transaction, no commit is applied. - If False (default), no commits are made, it relies on the current transaction. - If True, commits in batch. - If an integer, commits every N tree. To avoid memory issues, we recommend using incremental commit with large iterables.

Return type:

None

Modules

zodb

Contents