legendmeta package¶

A package to access legend-metadata in Python.

Submodules¶

legendmeta.catalog module¶

class legendmeta.catalog.Catalog(entries)¶

Bases: Catalog

Implementation of the JSONL metadata validity specification.

Create new instance of Catalog(entries,)

class Entry(valid_from, file)¶

Bases: Entry

Create new instance of Entry(valid_from, file)

static get(value)¶

static get_files(catalog_file, timestamp, category='all')¶

static read_from(file_name)¶

valid_for(timestamp, system='all', allow_none=False)¶

class legendmeta.catalog.Props¶

Bases: object

static add_to(props_a, props_b)¶

static read_from(sources, subst_pathvar=False, trim_null=False)¶

static subst_vars(props, var_values=None, ignore_missing=False)¶

static trim_null(props_a)¶

static write_to(file_name, obj, ftype: str | None = None)¶

class legendmeta.catalog.PropsStream¶

Bases: object

static get(value)¶

static read_from(file_name)¶

legendmeta.catalog.to_datetime(value)¶: Convert a LEGEND timestamp (or key) to datetime.datetime.

legendmeta.catalog.unix_time(value)¶

legendmeta.core module¶

class legendmeta.core.LegendMetadata(path: str | None = None, **kwargs)¶

Bases: TextDB

LEGEND metadata.

Class representing the LEGEND metadata repository with utilities for fast access.

Parameters:

path (str | None) – path to legend-metadata repository. If not existing, will attempt a git-clone through SSH. If None, legend-metadata will be cloned in a temporary directory (see tempfile.gettempdir()).
**kwargs – further keyword arguments forwarded to $TextDB.__init__$.

Construct a TextDB object.

Parameters:

path (str | None) – path to the directory containing the database.
lazy – whether a database scan should be performed at initialization time. if auto, be non-lazy only if working in a python interactive session.
hidden – ignore hidden (i.e. starting with “.”) files of directories.

_init_metadata_repo()¶: Clone legend-metadata, if not existing, and checkout default Git ref.

channelmap(on: str | datetime | None = None) → AttrsDict¶

Get a LEGEND channel map.

Aliases legend-metadata.hardware.configuration.channelmaps.on() and tries to merge the returned channel map with the detector database legend-metadata.hardware.detectors and the analysis channel map dataprod.config.on(…).analysis.

Warning

This method assumes legend-exp/legend-metadata has a certain layout. Might stop working if changes are made to the structure of the repository.

Examples

>>> from legendmeta import LegendMetadata
>>> from datetime import datetime
>>> channel = lmeta.channelmap(on=datetime.now()).V05267B
>>> channel.geometry.mass_in_g
2362.0
>>> channel.analysis.usability
'on'

See also

textdb.TextDB.on

Return type:: AttrsDict

checkout(git_ref: str) → None¶

Select a legend-metadata version.

reset() → None¶

Checkout legend-metadata to the default Git ref.

legendmeta.police module¶

legendmeta.police.len_nested(d: dict) → int¶

Recursively count keys in a dictionary.

Return type:: int

legendmeta.police.validate_dict_schema(adict: dict, template: dict, greedy: bool = True, typecheck=True, root_obj: str = '') → bool¶

Validate the format of a dictionary based on a template.

Prints error messages on the console. Returns false if dictionary is invalid.

Parameters:

adict (dict) – dictionary to analyze.
template (dict) – template dictionary.
greedy (bool) – if false, do not fail if the analyzed dictionary contains extra keys.
typecheck – if true, perform type checking.
root_obj (str) – key name (or path to) dictionary. Used for error printing.

Return type:

bool

legendmeta.police.validate_keys_recursive(adict: dict, template: dict) → bool¶

Return false if adict contains keys not in template.

Return type:: bool

legendmeta.police.validate_legend_channel_map() → bool¶

Validate list of LEGEND channel map files.

Invoked in CLI.

Return type:: bool

legendmeta.police.validate_legend_detector_db() → bool¶

Validate LEGEND detector database.

Invoked in CLI.

Return type:: bool

legendmeta.scdb_tables module¶

class legendmeta.scdb_tables.Base(**kwargs: Any)¶

Bases: DeclarativeBase

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>¶

metadata: ClassVar[MetaData] = MetaData()¶: Refers to the _schema.MetaData collection that will be used for new _schema.Table objects.

See also

Accessing Table and Metadata

registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>¶: Refers to the _orm.registry in use where new _orm.Mapper objects will be associated.

class legendmeta.scdb_tables.DiodeConfMon(**kwargs)¶

Bases: Base

Configuration parameters of HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

confid: Mapped[int]¶

crate: Mapped[int]¶

iset: Mapped[float]¶

pwkill: Mapped[str]¶

pwon: Mapped[str]¶

rdown: Mapped[int]¶

rup: Mapped[int]¶

slot: Mapped[int]¶

trip: Mapped[float]¶

tstamp: Mapped[datetime]¶

vmax: Mapped[int]¶

vset: Mapped[float]¶

class legendmeta.scdb_tables.DiodeInfo(**kwargs)¶

Bases: Base

Static information about HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

crate: Mapped[int]¶

group: Mapped[str]¶

itol: Mapped[float]¶

iupd: Mapped[float]¶

label: Mapped[str]¶

slot: Mapped[int]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vtol: Mapped[float]¶

vupd: Mapped[float]¶

class legendmeta.scdb_tables.DiodeSnap(**kwargs)¶

Bases: Base

Monitored parameters of HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

almask: Mapped[int]¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

crate: Mapped[int]¶

imon: Mapped[float]¶

slot: Mapped[int]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vmon: Mapped[float]¶

class legendmeta.scdb_tables.MuonConfMon(**kwargs)¶

Bases: Base

Configuration parameters of PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

confid: Mapped[int]¶

crate: Mapped[int]¶

iset: Mapped[float]¶

pwkill: Mapped[str]¶

pwon: Mapped[str]¶

rdown: Mapped[int]¶

rup: Mapped[int]¶

slot: Mapped[int]¶

trip: Mapped[float]¶

tstamp: Mapped[datetime]¶

vmax: Mapped[int]¶

vset: Mapped[float]¶

class legendmeta.scdb_tables.MuonInfo(**kwargs)¶

Bases: Base

Static information about PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

crate: Mapped[int]¶

group: Mapped[str]¶

itol: Mapped[float]¶

iupd: Mapped[float]¶

label: Mapped[str]¶

slot: Mapped[int]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vtol: Mapped[float]¶

vupd: Mapped[float]¶

class legendmeta.scdb_tables.MuonSnap(**kwargs)¶

Bases: Base

Monitored parameters of PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

almask: Mapped[int]¶

asdict() → AttrsDict¶

Return type:: AttrsDict

channel: Mapped[int]¶

crate: Mapped[int]¶

imon: Mapped[float]¶

slot: Mapped[int]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vmon: Mapped[float]¶

class legendmeta.scdb_tables.SiPMConfMon(**kwargs)¶

Bases: Base

Configuration parameters of SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

board: Mapped[int]¶

channel: Mapped[int]¶

confid: Mapped[int]¶

iset: Mapped[float]¶

tstamp: Mapped[datetime]¶

vset: Mapped[float]¶

class legendmeta.scdb_tables.SiPMInfo(**kwargs)¶

Bases: Base

Static information about SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

asdict() → AttrsDict¶

Return type:: AttrsDict

board: Mapped[int]¶

channel: Mapped[int]¶

group: Mapped[str]¶

itol: Mapped[float]¶

iupd: Mapped[float]¶

label: Mapped[str]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vtol: Mapped[float]¶

vupd: Mapped[float]¶

class legendmeta.scdb_tables.SiPMSnap(**kwargs)¶

Bases: Base

Monitored parameters of SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶

almask: Mapped[int]¶

asdict() → AttrsDict¶

Return type:: AttrsDict

board: Mapped[int]¶

channel: Mapped[int]¶

imon: Mapped[float]¶

status: Mapped[int]¶

tstamp: Mapped[datetime]¶

vmon: Mapped[float]¶

legendmeta.slowcontrol module¶

class legendmeta.slowcontrol.LegendSlowControlDB(connect=False)¶

Bases: object

A class for interacting with the LEGEND Slow Control database.

The LEGEND Slow Control system relies on a PostgreSQL database living on legend-sc.lngs.infn.it. The aim of the LegendSlowControlDB class is to simplify access to the database from Python.

connect(host: str = 'localhost', port: int = 5432, password: str | None = None) → None¶

Establish a connection to the database.

Authentication is attempted with the read-only user scuser on a database named scdb.

Parameters:

host (str) – database host. Can be a hostname (localhost, legend-sc.lngs.infn.it, etc.) or an IP address.
port (int) – port through which the database should be contacted.
password (str | None) – password for user scuser of the scdb database. May be found on LEGEND’s internal documentation (e.g. the Wiki web pages). If None, uses the value of the $LEGEND_SCDB_PW shell variable.

Examples

If the Slow Control database connection is forwarded to a local machine (port 6942) (through e.g. an SSH tunnel), use:

>>> scdb = LegendSlowControlDB()
>>> scdb.connect("localhost", port=6942, password="···")

Alternatively to giving the password to connect(), it can be stored in the $LEGEND_SCDB_PW shell variable (in e.g. .bashrc):

~/.bashrc¶

export LEGEND_SCDB_PW="···"

Then:

>>> scdb.connect("localhost", port=6942)

dataframe(expr: str | Select) → DataFrame¶

Query the database and return a dataframe holding the result.

Parameters:: expr (str | Select) – SQL table name, select SQL command text or SQLAlchemy selectable object.
Return type:: DataFrame

Examples

SQL select syntax text or table name:

>>> scdb.dataframe("SELECT channel, vmon FROM diode_snap LIMIT 3")
   channel    vmon
0        2  2250.0
1        1  3899.4
2        5  1120.2

>>> scdb.dataframe("diode_conf_mon")
      confid  crate  slot  channel    vset  iset  rup  rdown  trip  vmax pwkill pwon                    tstamp
0         15      0     0        0  4000.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
1         15      0     0        1  4300.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
2         15      0     0        2  4200.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
...

sqlalchemy.sql.Select object:

>>> import sqlalchemy as sql
>>> from legendmeta.slowcontrol import DiodeSnap
>>> scdb.dataframe(sql.select(DiodeSnap.channel, DiodeSnap.vmon).limit(3))
   channel    vmon
0        2  2250.0
1        1  3899.4
2        5  1120.2

See also

pandas.read_sql

disconnect() → None¶

Disconnect from the database.

get_columns(table: str) → list[str]¶

Get columns available on table in the database.

Return type:: list[str]

get_tables() → list[str]¶

Get tables available in the database.

Return type:: list[str]

make_session() → Session¶

Open and return a new Session object for executing database operations.

Examples

>>> import sqlalchemy as sql
>>> from legendmeta.slowcontrol import DiodeSnap
>>> session = scdb.make_session()
>>> result = session.execute(sql.select(DiodeSnap.channel, DiodeSnap.imon).limit(3))
>>> result.all()
[(2, 0.0007), (1, 0.0001), (5, 5e-05)]

See also

SQLAlchemy documentation

Return type:: Session

status(channel: dict, on: str | datetime | None = None, system: str | None = None) → dict¶

Query status of a LEGEND DAQ channel.

Collect all the relevant information about the status of a channel at a certain time from the Slow Control database, based on the channel type (i.e. germanium detector, SiPM or PMT).

Parameters:

channel (dict) – this dictionary must contain information about the channel hardware configuration. Typically a LEGEND channel map entry (obtainable, for example, with core.LegendMetadata.channelmap()).
on (str | datetime | None) – time at which the status is requested.
system ("geds", "spms", "pmts", ...) – system the channel belong to. this information is used to ask the Slow Control database the right questions. If None will try to determine it from the available metadata.

Return type:

dict

Examples

>>> ts = datetime(...) # or LEGEND cycle timestamp
>>> my_bege = lmeta.channelmap(on=ts).B00089B
>>> scdb.status(my_bege, on=ts)
{'vmon': 3399.9,
 'imon': 0.00015,
 'status': 1,
 'vset': 3400.0,
 'iset': 6.0,
 ...

Warning

This class method assumes a certain structure for legend-metadata. Might stop working if that structure is altered.

legendmeta.textdb module¶

class legendmeta.textdb.AttrsDict(value: dict | None = None)¶

Bases: dict

Access dictionary items as attributes.

Examples

>>> d = AttrsDict({"key1": {"key2": 1}})
>>> d.key1.key2
1
>>> d1 = AttrsDict()
>>> d1["a"] = 1
>>> d1.a
1

Construct an AttrsDict object.

Note

The input dictionary is copied.

Parameters:: value (dict | None) – a dict object to initialize the instance with.

group(label: str) → AttrsDict¶

Group dictionary according to a label.

This is equivalent to map() with unique set to False.

Parameters:: label (str) – name (key) at which the new label can be found. If nested in dictionaries, use . to separate levels, e.g. level1.level2.label.
Return type:: AttrsDict

Examples

>>> d = AttrsDict({
...   "a": {
...     "type": "A",
...     "data": 1
...   },
...   "b": {
...     "type": "A",
...     "data": 2
...   },
...   "c": {
...     "type": "B",
...     "data": 3
...   },
... })
>>> d.group("type").keys()
dict_keys(['A', 'B'])
>>> d.group("type").A.values()
dict_values([{'type': 'A', 'data': 1}, {'type': 'A', 'data': 2}])
>>> d.group("type").B.values()
dict_values([{'type': 'B', 'data': 3}])
>>> d.group("type").A.map("data")[1]
{'type': 'A', 'data': 1}

See also

map

map(label: str, unique: bool = True) → AttrsDict¶

Remap dictionary according to an alternative unique label.

Loop over keys in the first level and search for key named label in their values. If label is found and its value newid is unique, create a mapping between newid and the first-level dictionary obj. If label is of the form key.label, label will be searched in a dictionary keyed by key. If the label is unique a dictionary of dictionaries will be returned, if not unique and unique is false, a dictionary will be returned where each entry is a dictionary of dictionaries keyed by an arbitrary integer.

Parameters:

label (str) – name (key) at which the new label can be found. If nested in dictionaries, use . to separate levels, e.g. level1.level2.label.
unique (bool) – bool specifying whether only unique keys are allowed. If true will raise an error if the specified key is not unique.

Return type:

AttrsDict

Examples

>>> d = AttrsDict({
...   "a": {
...     "id": 1,
...     "group": {
...       "id": 3,
...     },
...     "data": "x"
...   },
...   "b": {
...     "id": 2,
...     "group": {
...       "id": 4,
...     },
...     "data": "y"
...   },
... })
>>> d.map("id")[1].data == "x"
True
>>> d.map("group.id")[4].data == "y"
True

Note

No copy is performed, the returned dictionary is made of references to the original objects.

Warning

The result is cached internally for fast access after the first call. If the dictionary is modified, the cache gets cleared.

legendmeta.textdb.JsonDB(*args, **kwargs)¶

class legendmeta.textdb.TextDB(path: str | Path, lazy: str | bool = False, hidden: bool = False)¶

Bases: object

A simple text file database.

The database is represented on disk by a collection of text files arbitrarily scattered in a filesystem. Subdirectories are also TextDB objects. In memory, the database is represented as an AttrsDict.

Currently supported file formats are JSON and YAML.

Tip

For large databases, a basic “lazy” mode is available. In this case, no global scan of the filesystem is performed at initialization time. Once a file is queried, it is also cached in the internal store for faster access. Caution, this option is for advanced use (see warning message below).

Warning

A manual call to scan() is needed before most class methods (e.g. iterating on the database files) can be properly used.

Examples

>>> from legendmeta.jsondb import TextDB
>>> jdb = TextDB("path/to/dir")
>>> jdb["file1.json"]  # is a dict
>>> jdb["file1.yaml"]  # is a dict
>>> jdb["file1"]  # also works
>>> jdb["dir1"]  # TextDB instance
>>> jdb["dir1"]["file1"]  # nested file
>>> jdb["dir1/file1"]  # also works
>>> jdb.dir1.file # keys can be accessed as attributes

Construct a TextDB object.

Parameters:

path (str | Path) – path to the directory containing the database.
lazy (str | bool) – whether a database scan should be performed at initialization time. if auto, be non-lazy only if working in a python interactive session.
hidden (bool) – ignore hidden (i.e. starting with “.”) files of directories.

group(label: str) → AttrsDict¶

Group dictionary according to a second unique label.

See also

AttrsDict.group

Warning

If the database is lazy, you must call scan() in advance to populate it, otherwise groupings cannot be created.

Return type:: AttrsDict

items() → Iterator[str, TextDB | AttrsDict | list]¶

Return type:: Iterator[str, TextDB | AttrsDict | list]

keys() → list[str]¶

Return type:: list[str]

map(label: str, unique: bool = True) → AttrsDict¶

Remap dictionary according to a second unique label.

See also

AttrsDict.map

Warning

If the database is lazy, you must call scan() in advance to populate it, otherwise mappings cannot be created.

Return type:: AttrsDict

on(timestamp: str | datetime, pattern: str | None = None, system: str = 'all') → AttrsDict | list¶

Query database in time[, file pattern, system].

A (only one) valid validity.jsonl file must exist in the directory to specify a validity mapping. This functionality relies on the catalog.Catalog class.

The JSONL specification is documented at this link.

The special $_ string is expanded to the directory containing the text files.

Parameters:

timestamp (str | datetime) – a datetime object or a string matching the pattern YYYYmmddTHHMMSSZ.
pattern (str | None) – query by filename pattern.
system ('all', 'phy', 'cal', 'lar', ...) – query only a data taking “system”.

Return type:

AttrsDict | list

scan(recursive: bool = True, subdir: str = '.') → None¶

Populate the database by walking the filesystem.

Parameters:

recursive (bool) – if True, recurse subdirectories.
subdir (str) – restrict scan to path relative to the database location.

legendmeta.utils module¶

legendmeta.utils.load_dict(fname: str, ftype: str | None = None) → dict¶

Load a text file as a Python dict.

Return type:: dict

legendmeta.utils.write_dict(fname: str, obj: dict, ftype: str | None = None) → dict¶

Load a text file as a Python dict.

Return type:: dict