legendmeta package¶
A package to access legend-metadata in Python.
Submodules¶
legendmeta.core module¶
- class legendmeta.core.MetadataRepository(path=None, repo_url='', env_var='', default_dir_name='', **kwargs)¶
Bases:
TextDBBase class for git-based metadata repositories.
Class representing a metadata repository with utilities for fast access via git.
If no valid path to an existing metadata directory is provided, will attempt to clone the repository via SSH and git-checkout the latest stable tag (vM.m.p format).
Note
This class is designed to be subclassed, not instantiated directly. Derived classes should provide the required repository configuration (repo_url, env_var, default_dir_name) via
super().__init__().- Parameters:
path (str | None) – path to metadata repository. If not existing, will attempt a git-clone through SSH. If
None, metadata will be cloned in a temporary directory (seetempfile.gettempdir()).repo_url (str) – URL of the git repository to clone (e.g., “git@github.com:org/repo”).
env_var (str) – name of the environment variable to check for repository path.
default_dir_name (str) – default directory name for cloning in temp directory.
**kwargs – further keyword arguments forwarded to
TextDB.__init__.
Construct a
TextDBobject.- Parameters:
path (str | None) – path to the directory containing the database.
lazy – whether a database scan should be performed at initialization time. if
auto, be non-lazy only if working in a python interactive session.hidden – ignore hidden (i.e. starting with “.”) files of directories.
- _except_if_not_git_repo()¶
- _init_metadata_repo()¶
Clone metadata repository, if not existing, and checkout latest stable tag.
- checkout(git_ref, rescan=True)¶
Select a metadata repository version.
- property latest_stable_tag: Version | None¶
Latest stable metadata tag (i.e. strictly numeric vM.m.p)
- show_metadata_version()¶
Logs version info for metadata repository and all its submodules.
legendmeta.hadesmetadata module¶
- class legendmeta.hadesmetadata.HadesMetadata(path=None, **kwargs)¶
Bases:
MetadataRepositoryHADES metadata.
Class representing the HADES metadata repository with utilities for fast access.
If no valid path to an existing hades-metadata directory is provided, will attempt to clone https://github.com/legend-exp/hades-metadata via SSH and git-checkout the latest stable tag (vM.m.p format).
- Parameters:
path (str | None) – path to hades-metadata repository. If not existing, will attempt a git-clone through SSH. If
None, hades-metadata will be cloned in a temporary directory (seetempfile.gettempdir()).**kwargs – further keyword arguments forwarded to
TextDB.__init__.
Construct a
TextDBobject.- Parameters:
path (str | None) – path to the directory containing the database.
lazy – whether a database scan should be performed at initialization time. if
auto, be non-lazy only if working in a python interactive session.hidden – ignore hidden (i.e. starting with “.”) files of directories.
legendmeta.legendmetadata module¶
- class legendmeta.legendmetadata.LegendMetadata(path=None, **kwargs)¶
Bases:
MetadataRepositoryLEGEND metadata.
Class representing the LEGEND metadata repository with utilities for fast access.
If no valid path to an existing legend-metadata directory is provided, will attempt to clone https://github.com/legend-exp/legend-metadata via SSH and git-checkout the latest stable tag (vM.m.p format).
- Parameters:
path (str | None) – path to legend-metadata repository. If not existing, will attempt a git-clone through SSH. If
None, legend-metadata will be cloned in a temporary directory (seetempfile.gettempdir()).**kwargs – further keyword arguments forwarded to
TextDB.__init__.
Construct a
TextDBobject.- Parameters:
path (str | None) – path to the directory containing the database.
lazy – whether a database scan should be performed at initialization time. if
auto, be non-lazy only if working in a python interactive session.hidden – ignore hidden (i.e. starting with “.”) files of directories.
- channelmap(on=None, system='all', skip_version_check=False)¶
Get a LEGEND channel map.
Aliases
legend-metadata.hardware.configuration.channelmaps.on()and tries to merge the returned channel map with the detector database legend-metadata.hardware.detectors and the analysis channel map dataprod.config.on(…).analysis.- Parameters:
on (str | datetime | None) – a
datetimeobject or a string matching the patternYYYYmmddTHHMMSSZ.system ('all', 'phy', 'cal', 'lar', ...) – query only a data taking “system”.
skip_version_check (bool) – if
True, skip the git version check and assume the latest metadata structure. This is useful when working with non-git repositories (e.g., test data).
- Return type:
Warning
This method assumes
legend-exp/legend-metadatahas a certain layout. Might stop working if changes are made to the structure of the repository.Examples
>>> from legendmeta import LegendMetadata >>> from datetime import datetime >>> channel = lmeta.channelmap(on=datetime.now()).V05267B >>> channel.geometry.mass_in_g 2362.0 >>> channel.analysis.usability 'on'
See also
textdb.TextDB.on
legendmeta.police module¶
- legendmeta.police._check_cal_override_runs(file, overridden_runs, verbose=True)¶
Check that no overridden run appears in a cal_groupings file.
- legendmeta.police._check_chmap_key_name(key, entry, verbose=True)¶
Return True if entry’s ‘name’ field is absent or matches the dict key.
- Return type:
- legendmeta.police._fix_groupings_file(file)¶
Sort a groupings file in place. Returns True if the file was modified.
- Return type:
- legendmeta.police._fix_status_files(validity_file)¶
Sort channel entry keys in all status YAML files beside the validity file.
Returns True if any file was modified.
- Return type:
- legendmeta.police._get_overridden_runs(run_override_entries, runinfo)¶
Return (period, run) pairs whose cal falls inside a run_override window.
The run_override file defines windows of time during which certain replacement cal files are applied (non-empty
applylist). Any run whosecal.start_keyfalls within such a window (i.e. the timestamp is ≥ the window start and < the next reset timestamp) is considered overridden. A single window can cover multiple calibrations.
- legendmeta.police._needs_reorder(a, b)¶
Return True if key order differs (recursively) between a and b.
- Return type:
- legendmeta.police._run_sort_key(spec)¶
Sort key for a run spec (individual or range): the starting run string.
- Return type:
- legendmeta.police._sort_groupings_data(data)¶
Return a copy of groupings data with all keys sorted, default first.
- Return type:
- legendmeta.police._sort_groupings_groups(groups)¶
Return a copy of a groups dict with groups and periods/runs sorted.
- Return type:
- legendmeta.police._sort_status_entry(entry)¶
Return a copy of a channel status entry with keys in canonical order.
- Return type:
- legendmeta.police._validate_groupings_file(file, group_prefix, verbose=True)¶
Shared validation logic for cal/phy groupings files.
- legendmeta.police._validate_run_spec(runs, location, verbose=True)¶
Validate a run specification: string range or list of runs/ranges.
- Return type:
- legendmeta.police.validate_cal_groupings()¶
Validate LEGEND calibration groupings files.
Invoked in CLI. Expects run_override.yaml and runinfo.yaml to be in the same directory as each groupings file.
- legendmeta.police.validate_dict_schema(adict, template, greedy=True, typecheck=True, root_obj='', verbose=True)¶
Validate the format of a dictionary based on a template.
Prints error messages on the console. Returns false if dictionary is invalid.
- Parameters:
adict (dict) – dictionary to analyze.
template (dict) – template dictionary.
greedy (bool) – if false, do not fail if the analyzed dictionary contains extra keys.
typecheck – if true, perform type checking.
root_obj (str) – key name (or path to) dictionary. Used for error printing.
verbose (bool) – if false, do not print error messages.
- Return type:
- legendmeta.police.validate_keys_recursive(adict, template, verbose=True)¶
Return false if adict contains keys not in template.
- Return type:
- legendmeta.police.validate_legend_channel_map()¶
Validate list of LEGEND channel map files.
Invoked in CLI.
- Return type:
- legendmeta.police.validate_legend_detector_db()¶
Validate LEGEND detector database.
Invoked in CLI.
- Return type:
- legendmeta.police.validate_phy_groupings()¶
Validate LEGEND physics groupings files.
Invoked in CLI.
- legendmeta.police.validate_statuses()¶
Validate LEGEND status files.
Invoked in CLI. Accepts validity files; derives the status directory from each.
- legendmeta.police.validate_validity()¶
legendmeta.scdb_tables module¶
- class legendmeta.scdb_tables.Base(**kwargs)¶
Bases:
DeclarativeBaseA simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>¶
- metadata: ClassVar[MetaData] = MetaData()¶
Refers to the
_schema.MetaDatacollection that will be used for new_schema.Tableobjects.See also
- registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>¶
Refers to the
_orm.registryin use where new_orm.Mapperobjects will be associated.
- class legendmeta.scdb_tables.DiodeConfMon(**kwargs)¶
Bases:
BaseConfiguration parameters of HPGe detectors.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.DiodeInfo(**kwargs)¶
Bases:
BaseStatic information about HPGe detectors.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.DiodeSnap(**kwargs)¶
Bases:
BaseMonitored parameters of HPGe detectors.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.MuonConfMon(**kwargs)¶
Bases:
BaseConfiguration parameters of PMTs from the muon veto.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwkill': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'pwon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rdown': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'rup': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'trip': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmax': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.MuonInfo(**kwargs)¶
Bases:
BaseStatic information about PMTs from the muon veto.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.MuonSnap(**kwargs)¶
Bases:
BaseMonitored parameters of PMTs from the muon veto.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'crate': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'slot': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.SiPMConfMon(**kwargs)¶
Bases:
BaseConfiguration parameters of SiPMs from the LAr instrumentation.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'confid': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vset': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.SiPMInfo(**kwargs)¶
Bases:
BaseStatic information about SiPMs from the LAr instrumentation.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'group': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'itol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'iupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'label': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vtol': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vupd': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
- class legendmeta.scdb_tables.SiPMSnap(**kwargs)¶
Bases:
BaseMonitored parameters of SiPMs from the LAr instrumentation.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- _sa_class_manager = {'almask': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'board': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'channel': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'imon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'status': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'tstamp': <sqlalchemy.orm.attributes.InstrumentedAttribute object>, 'vmon': <sqlalchemy.orm.attributes.InstrumentedAttribute object>}¶
- tstamp: Mapped[datetime]¶
legendmeta.slowcontrol module¶
- class legendmeta.slowcontrol.LegendSlowControlDB(connect=False)¶
Bases:
objectA class for interacting with the LEGEND Slow Control database.
The LEGEND Slow Control system relies on a PostgreSQL database living on
legend-sc.lngs.infn.it. The aim of theLegendSlowControlDBclass is to simplify access to the database from Python.- connect(host='localhost', port=5432, password=None, db_name='scdb')¶
Establish a connection to the database.
Authentication is attempted with the read-only user
scuser.- Parameters:
host (str) – database host. Can be a hostname (
localhost,legend-sc.lngs.infn.it, etc.) or an IP address.port (int) – port through which the database should be contacted.
password (str | None) – password for user
scuserof thescdbdatabase. May be found on LEGEND’s internal documentation (e.g. the Wiki web pages). IfNone, uses the value of the$LEGEND_SCDB_PWshell variable.db_name (str) – name of the database, default is
scdb:scdbis the current database, active since 2024-09-02;scdbL140is the database for the 142 kg deployment, from 2022-11-09 to 2024-08-28;scdbL60is for commissioning data with a 60 kg array, from 2021-12-23 to 2022-10-15.
Examples
If the Slow Control database connection is forwarded to a local machine (port 6942) (through e.g. an SSH tunnel), use:
>>> scdb = LegendSlowControlDB() >>> scdb.connect("localhost", port=6942, password="···")
Alternatively to giving the password to
connect(), it can be stored in the$LEGEND_SCDB_PWshell variable (in e.g..bashrc):~/.bashrc¶export LEGEND_SCDB_PW="···"
Then:
>>> scdb.connect("localhost", port=6942)
- dataframe(expr)¶
Query the database and return a dataframe holding the result.
- Parameters:
expr (str | Select) – SQL table name, select SQL command text or SQLAlchemy selectable object.
- Return type:
DataFrame
Examples
SQL select syntax text or table name:
>>> scdb.dataframe("SELECT channel, vmon FROM diode_snap LIMIT 3") channel vmon 0 2 2250.0 1 1 3899.4 2 5 1120.2
>>> scdb.dataframe("diode_conf_mon") confid crate slot channel vset iset rup rdown trip vmax pwkill pwon tstamp 0 15 0 0 0 4000.0 6.0 10 5 10.0 6000 KILL Dis 2022-10-07 13:49:56+00:00 1 15 0 0 1 4300.0 6.0 10 5 10.0 6000 KILL Dis 2022-10-07 13:49:56+00:00 2 15 0 0 2 4200.0 6.0 10 5 10.0 6000 KILL Dis 2022-10-07 13:49:56+00:00 ...
sqlalchemy.sql.Selectobject:>>> import sqlalchemy as sql >>> from legendmeta.slowcontrol import DiodeSnap >>> scdb.dataframe(sql.select(DiodeSnap.channel, DiodeSnap.vmon).limit(3)) channel vmon 0 2 2250.0 1 1 3899.4 2 5 1120.2
See also
- disconnect()¶
Disconnect from the database.
- make_session()¶
Open and return a new
Sessionobject for executing database operations.Examples
>>> import sqlalchemy as sql >>> from legendmeta.slowcontrol import DiodeSnap >>> session = scdb.make_session() >>> result = session.execute(sql.select(DiodeSnap.channel, DiodeSnap.imon).limit(3)) >>> result.all() [(2, 0.0007), (1, 0.0001), (5, 5e-05)]
See also
- Return type:
- status(channel, on=None, system=None)¶
Query status of a LEGEND DAQ channel.
Collect all the relevant information about the status of a channel at a certain time from the Slow Control database, based on the channel type (i.e. germanium detector, SiPM or PMT).
- Parameters:
channel (dict) – this dictionary must contain information about the channel hardware configuration. Typically a LEGEND channel map entry (obtainable, for example, with
core.LegendMetadata.channelmap()).on (str | datetime | None) – time at which the status is requested.
system ("geds", "spms", "pmts", ...) – system the channel belong to. this information is used to ask the Slow Control database the right questions. If
Nonewill try to determine it from the available metadata.
- Return type:
Examples
>>> ts = datetime(...) # or LEGEND cycle timestamp >>> my_bege = lmeta.channelmap(on=ts).B00089B >>> scdb.status(my_bege, on=ts) {'vmon': 3399.9, 'imon': 0.00015, 'status': 1, 'vset': 3400.0, 'iset': 6.0, ...
Warning
This class method assumes a certain structure for legend-metadata. Might stop working if that structure is altered.