legendmeta package

A package to access legend-metadata in Python.

Submodules

legendmeta.catalog module

legendmeta.catalog.to_datetime(value)

legendmeta.core module

class legendmeta.core.MetadataRepository(path=None, repo_url='', env_var='', default_dir_name='', **kwargs)

Bases: TextDB

Base class for git-based metadata repositories.

Class representing a metadata repository with utilities for fast access via git.

If no valid path to an existing metadata directory is provided, will attempt to clone the repository via SSH and git-checkout the latest stable tag (vM.m.p format).

Note

This class is designed to be subclassed, not instantiated directly. Derived classes should provide the required repository configuration (repo_url, env_var, default_dir_name) via super().__init__().

Parameters:

  • path (str | None) – path to metadata repository. If not existing, will attempt a git-clone through SSH. If None, metadata will be cloned in a temporary directory (see tempfile.gettempdir()).

  • repo_url (str) – URL of the git repository to clone (e.g., “git@github.com:org/repo”).

  • env_var (str) – name of the environment variable to check for repository path.

  • default_dir_name (str) – default directory name for cloning in temp directory.

  • **kwargs – further keyword arguments forwarded to TextDB.__init__.

Construct a TextDB object.

Parameters:

  • path (str | None) – path to the directory containing the database.

  • lazy – whether a database scan should be performed at initialization time. if auto, be non-lazy only if working in a python interactive session.

  • hidden – ignore hidden (i.e. starting with “.”) files of directories.

_except_if_not_git_repo()
_init_metadata_repo()

Clone metadata repository, if not existing, and checkout latest stable tag.

checkout(git_ref, rescan=True)

Select a metadata repository version.

property latest_stable_tag: Version | None

Latest stable metadata tag (i.e. strictly numeric vM.m.p)

show_metadata_version()

Logs version info for metadata repository and all its submodules.

legendmeta.hadesmetadata module

class legendmeta.hadesmetadata.HadesMetadata(path=None, **kwargs)

Bases: MetadataRepository

HADES metadata.

Class representing the HADES metadata repository with utilities for fast access.

If no valid path to an existing hades-metadata directory is provided, will attempt to clone https://github.com/legend-exp/hades-metadata via SSH and git-checkout the latest stable tag (vM.m.p format).

Parameters:

  • path (str | None) – path to hades-metadata repository. If not existing, will attempt a git-clone through SSH. If None, hades-metadata will be cloned in a temporary directory (see tempfile.gettempdir()).

  • **kwargs – further keyword arguments forwarded to TextDB.__init__.

Construct a TextDB object.

Parameters:

  • path (str | None) – path to the directory containing the database.

  • lazy – whether a database scan should be performed at initialization time. if auto, be non-lazy only if working in a python interactive session.

  • hidden – ignore hidden (i.e. starting with “.”) files of directories.

legendmeta.legendmetadata module

class legendmeta.legendmetadata.LegendMetadata(path=None, **kwargs)

Bases: MetadataRepository

LEGEND metadata.

Class representing the LEGEND metadata repository with utilities for fast access.

If no valid path to an existing legend-metadata directory is provided, will attempt to clone https://github.com/legend-exp/legend-metadata via SSH and git-checkout the latest stable tag (vM.m.p format).

Parameters:

  • path (str | None) – path to legend-metadata repository. If not existing, will attempt a git-clone through SSH. If None, legend-metadata will be cloned in a temporary directory (see tempfile.gettempdir()).

  • **kwargs – further keyword arguments forwarded to TextDB.__init__.

Construct a TextDB object.

Parameters:

  • path (str | None) – path to the directory containing the database.

  • lazy – whether a database scan should be performed at initialization time. if auto, be non-lazy only if working in a python interactive session.

  • hidden – ignore hidden (i.e. starting with “.”) files of directories.

channelmap(on=None, system='all')

Get a LEGEND channel map.

Aliases legend-metadata.hardware.configuration.channelmaps.on() and tries to merge the returned channel map with the detector database legend-metadata.hardware.detectors and the analysis channel map dataprod.config.on(…).analysis.

Parameters:

  • on (str | datetime | None) – a datetime object or a string matching the pattern YYYYmmddTHHMMSSZ.

  • system ('all', 'phy', 'cal', 'lar', ...) – query only a data taking “system”.

Return type:

AttrsDict

Warning

This method assumes legend-exp/legend-metadata has a certain layout. Might stop working if changes are made to the structure of the repository.

Examples

>>> from legendmeta import LegendMetadata
>>> from datetime import datetime
>>> channel = lmeta.channelmap(on=datetime.now()).V05267B
>>> channel.geometry.mass_in_g
2362.0
>>> channel.analysis.usability
'on'

See also

textdb.TextDB.on

legendmeta.police module

legendmeta.police.len_nested(d)

Recursively count keys in a dictionary.

Return type:

int

legendmeta.police.validate_dict_schema(adict, template, greedy=True, typecheck=True, root_obj='', verbose=True)

Validate the format of a dictionary based on a template.

Prints error messages on the console. Returns false if dictionary is invalid.

Parameters:

  • adict (dict) – dictionary to analyze.

  • template (dict) – template dictionary.

  • greedy (bool) – if false, do not fail if the analyzed dictionary contains extra keys.

  • typecheck – if true, perform type checking.

  • root_obj (str) – key name (or path to) dictionary. Used for error printing.

  • verbose (bool) – if false, do not print error messages.

Return type:

bool

legendmeta.police.validate_keys_recursive(adict, template, verbose=True)

Return false if adict contains keys not in template.

Return type:

bool

legendmeta.police.validate_legend_channel_map()

Validate list of LEGEND channel map files.

Invoked in CLI.

Return type:

bool

legendmeta.police.validate_legend_detector_db()

Validate LEGEND detector database.

Invoked in CLI.

Return type:

bool

legendmeta.police.validate_validity()

legendmeta.scdb_tables module

class legendmeta.scdb_tables.Base(**kwargs)

Bases: DeclarativeBase

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>
metadata: ClassVar[MetaData] = MetaData()

Refers to the _schema.MetaData collection that will be used for new _schema.Table objects.

See also

Accessing Table and Metadata

registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>

Refers to the _orm.registry in use where new _orm.Mapper objects will be associated.

class legendmeta.scdb_tables.DiodeConfMon(**kwargs)

Bases: Base

Configuration parameters of HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

channel: Mapped[int]
confid: Mapped[int]
crate: Mapped[int]
iset: Mapped[float]
pwkill: Mapped[str]
pwon: Mapped[str]
rdown: Mapped[int]
rup: Mapped[int]
slot: Mapped[int]
trip: Mapped[float]
tstamp: Mapped[datetime]
vmax: Mapped[int]
vset: Mapped[float]
class legendmeta.scdb_tables.DiodeInfo(**kwargs)

Bases: Base

Static information about HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

channel: Mapped[int]
crate: Mapped[int]
group: Mapped[str]
itol: Mapped[float]
iupd: Mapped[float]
label: Mapped[str]
slot: Mapped[int]
status: Mapped[int]
tstamp: Mapped[datetime]
vtol: Mapped[float]
vupd: Mapped[float]
class legendmeta.scdb_tables.DiodeSnap(**kwargs)

Bases: Base

Monitored parameters of HPGe detectors.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
almask: Mapped[int]
asdict()
Return type:

AttrsDict

channel: Mapped[int]
crate: Mapped[int]
imon: Mapped[float]
slot: Mapped[int]
status: Mapped[int]
tstamp: Mapped[datetime]
vmon: Mapped[float]
class legendmeta.scdb_tables.MuonConfMon(**kwargs)

Bases: Base

Configuration parameters of PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

channel: Mapped[int]
confid: Mapped[int]
crate: Mapped[int]
iset: Mapped[float]
pwkill: Mapped[str]
pwon: Mapped[str]
rdown: Mapped[int]
rup: Mapped[int]
slot: Mapped[int]
trip: Mapped[float]
tstamp: Mapped[datetime]
vmax: Mapped[int]
vset: Mapped[float]
class legendmeta.scdb_tables.MuonInfo(**kwargs)

Bases: Base

Static information about PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

channel: Mapped[int]
crate: Mapped[int]
group: Mapped[str]
itol: Mapped[float]
iupd: Mapped[float]
label: Mapped[str]
slot: Mapped[int]
status: Mapped[int]
tstamp: Mapped[datetime]
vtol: Mapped[float]
vupd: Mapped[float]
class legendmeta.scdb_tables.MuonSnap(**kwargs)

Bases: Base

Monitored parameters of PMTs from the muon veto.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
almask: Mapped[int]
asdict()
Return type:

AttrsDict

channel: Mapped[int]
crate: Mapped[int]
imon: Mapped[float]
slot: Mapped[int]
status: Mapped[int]
tstamp: Mapped[datetime]
vmon: Mapped[float]
class legendmeta.scdb_tables.SiPMConfMon(**kwargs)

Bases: Base

Configuration parameters of SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

board: Mapped[int]
channel: Mapped[int]
confid: Mapped[int]
iset: Mapped[float]
tstamp: Mapped[datetime]
vset: Mapped[float]
class legendmeta.scdb_tables.SiPMInfo(**kwargs)

Bases: Base

Static information about SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
asdict()
Return type:

AttrsDict

board: Mapped[int]
channel: Mapped[int]
group: Mapped[str]
itol: Mapped[float]
iupd: Mapped[float]
label: Mapped[str]
status: Mapped[int]
tstamp: Mapped[datetime]
vtol: Mapped[float]
vupd: Mapped[float]
class legendmeta.scdb_tables.SiPMSnap(**kwargs)

Bases: Base

Monitored parameters of SiPMs from the LAr instrumentation.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

_sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}
almask: Mapped[int]
asdict()
Return type:

AttrsDict

board: Mapped[int]
channel: Mapped[int]
imon: Mapped[float]
status: Mapped[int]
tstamp: Mapped[datetime]
vmon: Mapped[float]

legendmeta.slowcontrol module

class legendmeta.slowcontrol.LegendSlowControlDB(connect=False)

Bases: object

A class for interacting with the LEGEND Slow Control database.

The LEGEND Slow Control system relies on a PostgreSQL database living on legend-sc.lngs.infn.it. The aim of the LegendSlowControlDB class is to simplify access to the database from Python.

connect(host='localhost', port=5432, password=None, db_name='scdb')

Establish a connection to the database.

Authentication is attempted with the read-only user scuser.

Parameters:

  • host (str) – database host. Can be a hostname (localhost, legend-sc.lngs.infn.it, etc.) or an IP address.

  • port (int) – port through which the database should be contacted.

  • password (str | None) – password for user scuser of the scdb database. May be found on LEGEND’s internal documentation (e.g. the Wiki web pages). If None, uses the value of the $LEGEND_SCDB_PW shell variable.

  • db_name (str) – name of the database, default is scdb: scdb is the current database, active since 2024-09-02; scdbL140 is the database for the 142 kg deployment, from 2022-11-09 to 2024-08-28; scdbL60 is for commissioning data with a 60 kg array, from 2021-12-23 to 2022-10-15.

Examples

If the Slow Control database connection is forwarded to a local machine (port 6942) (through e.g. an SSH tunnel), use:

>>> scdb = LegendSlowControlDB()
>>> scdb.connect("localhost", port=6942, password="···")

Alternatively to giving the password to connect(), it can be stored in the $LEGEND_SCDB_PW shell variable (in e.g. .bashrc):

~/.bashrc
export LEGEND_SCDB_PW="···"

Then:

>>> scdb.connect("localhost", port=6942)
dataframe(expr)

Query the database and return a dataframe holding the result.

Parameters:

expr (str | Select) – SQL table name, select SQL command text or SQLAlchemy selectable object.

Return type:

DataFrame

Examples

SQL select syntax text or table name:

>>> scdb.dataframe("SELECT channel, vmon FROM diode_snap LIMIT 3")
   channel    vmon
0        2  2250.0
1        1  3899.4
2        5  1120.2

>>> scdb.dataframe("diode_conf_mon")
      confid  crate  slot  channel    vset  iset  rup  rdown  trip  vmax pwkill pwon                    tstamp
0         15      0     0        0  4000.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
1         15      0     0        1  4300.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
2         15      0     0        2  4200.0   6.0   10      5  10.0  6000   KILL  Dis 2022-10-07 13:49:56+00:00
...

sqlalchemy.sql.Select object:

>>> import sqlalchemy as sql
>>> from legendmeta.slowcontrol import DiodeSnap
>>> scdb.dataframe(sql.select(DiodeSnap.channel, DiodeSnap.vmon).limit(3))
   channel    vmon
0        2  2250.0
1        1  3899.4
2        5  1120.2

See also

pandas.read_sql

disconnect()

Disconnect from the database.

get_columns(table)

Get columns available on table in the database.

Return type:

list[str]

get_tables()

Get tables available in the database.

Return type:

list[str]

make_session()

Open and return a new Session object for executing database operations.

Examples

>>> import sqlalchemy as sql
>>> from legendmeta.slowcontrol import DiodeSnap
>>> session = scdb.make_session()
>>> result = session.execute(sql.select(DiodeSnap.channel, DiodeSnap.imon).limit(3))
>>> result.all()
[(2, 0.0007), (1, 0.0001), (5, 5e-05)]

See also

SQLAlchemy documentation

Return type:

Session

status(channel, on=None, system=None)

Query status of a LEGEND DAQ channel.

Collect all the relevant information about the status of a channel at a certain time from the Slow Control database, based on the channel type (i.e. germanium detector, SiPM or PMT).

Parameters:

  • channel (dict) – this dictionary must contain information about the channel hardware configuration. Typically a LEGEND channel map entry (obtainable, for example, with core.LegendMetadata.channelmap()).

  • on (str | datetime | None) – time at which the status is requested.

  • system ("geds", "spms", "pmts", ...) – system the channel belong to. this information is used to ask the Slow Control database the right questions. If None will try to determine it from the available metadata.

Return type:

dict

Examples

>>> ts = datetime(...) # or LEGEND cycle timestamp
>>> my_bege = lmeta.channelmap(on=ts).B00089B
>>> scdb.status(my_bege, on=ts)
{'vmon': 3399.9,
 'imon': 0.00015,
 'status': 1,
 'vset': 3400.0,
 'iset': 6.0,
 ...

Warning

This class method assumes a certain structure for legend-metadata. Might stop working if that structure is altered.