Entry#

The Entry is the core class of metacatalog. It represents the core logical unit of the meta data model. In principle, an Entry needs a first Author, a title, position and a license to describe one type of environmental variable. It can hold a reference and interface to the actual data. If a supported data format is used, Entry can load the data.

class metacatalog.models.entry.Entry(**kwargs)#

The Entry is the main entity in metacatalog. An object instance models a set of metadata needed to store and manage a datasource. The Entry is not the actual data. The Entry is designed to store all necessary information to be exportable in ISO19115 in the scope of metacatalog. That means, Properties which are always the same across metacatalog, or can be derived from the actual implementation, are not part of an Entry.

id#

Unique id of the record. If not specified, the database will assign it.

Type:

int

uuid#

New in version 0.1.9.

Version 4 UUID string to identify the Entry across installations. This field is read-only and will be assigned on creation. It is primarily used to export Entry into ISO19115 metadata.

Type:

str

title#

A full title (512) to describe the datasource as well as possible. The truncated title (first 25 signs) is usually used to print an Entry object to the console.

Type:

str

abstract#

Full abstract of the datasource. The abstract should include all necessary information that is needed to fully understand the data.

Type:

str

external_id#

Any kind of OID that was used to identify the data in the first place. Usually an unque ID field of other data-storage solutions. The exernal_id is only stored for reference reasons.

Type:

str

location#

Changed in version 0.6.1.

A POINT location should be specified here if there is a physical measurement point that is different from the centroid of the spatial extent (e.g., discharge measurement with the extent of the catchment).Otherwise, Datasource.spatial_scale.extent should be used to specify the location of the measured data.

The location as a POINT Geometry in unprojected WGS84 (EPSG: 4326). The location is primarily used to show all Entry objects on a map, or perform geo-searches. The location can be passed as WKT or a tuple of (x, y) coordinates. Note that it will be returned and stored as WKB. The output value will be reworked in a future release

Type:

str, tuple

creation#

Following the ISO19115 the creation date is referring to the creation date of the data resource described by the Entry, not the Entry itself. If creation date is not set, it is assumed, that yet no data resource is connected to the Entry.

Type:

datetime.datetime

end#

The last date the data source described by this Entry has data for. The end date is not ISO19115-compliant and will be reworked.

Type:

datetime.datimetime

version#

The version of this Entry. Usually metacatalog will handle the version itself and there is not need to set the version manually.

Type:

int

latest_version_id#

Foreign key to Entry.id. This key is self-referencing the another Entry. This has to be set if the current Entry is not the latest one. If latest_version_id is None, the Entry is the most recent one and database operations that find multiple entries will in a future release filter to ‘version duplicates’.

Type:

int

is_partial#

New in version 0.1.10.

If an Entry is partial, it is not self-contained and has to be part of a EntryGroup of type composite.

Note

To make it possbile to add partial Entrys via the models submodule, The Entry class itself will not check integrity. This has to be done on adding partial Entry records, or by checking the database

Type:

bool

comment#

Arbitrary free-text comment to the Entry

Type:

str

citation#

New in version 0.1.13.

Citation informatio for this Entry. Note, that metacatalog does not assign DOIs and thus a citation is only useful if the associated data has a DOI and the bibliographic information applies to the Entry as well. .. note:

Metacatalog does not manage bibliography. Thus it is highly
recommended to use thrid party software for management and only
export the reference to the resource in a common citation style.
Type:

str

license#

Data License associated to the data and the metadata. You can pass the License itself, or use the license_id attribute.

Type:

metacatalog.models.License

license_id#

Foreign key to the data license.

Type:

int

author#

Person that acts as first author for the given entry. Only one first author is possible, co-authors can be requested from either the contributors list or the authors property. author is a property and setting a new author using this property is not supported.

Type:

metacatalog.models.Person

authors#

List of Person. The first element is the first author, see author. The others are Person of 'coAuthor' ``. The list of authors is sorted by the ``order attribute. authors is a property and setting a new list of authors using this property is not supported.

Type:

list

Note

One Entry object instance is always described by exactly one variable. If a datasource is a composite of many datasources, there are two strategies. Either a new table can be implemented and an abstract Variable be added. This is done with Eddy-Covariance data. Secondly, Each variable of the datasource can be represented by its own Entry, which get then grouped by an EntryGroup of EntryGroupType 'composite'.

See also

EntryGroup `EntryGroupType

add_details(details=None, commit: bool = False, **kwargs) → None#

Adds arbitrary key-value pairs to this entry.

Parameters:

  • details (list) –

    New in version 0.1.8.

    List of dict of structure:

    [{
    'key': '',
    'value': '',
    'description': ''
}]

    where the description is optional and can be omitted. If no descriptions are passed at all, you can also use **kwargs to pass key=value pairs.

  • commit (bool) – If True, the Entry session will be added to the current session and the transaction is commited. Can have side-effects. Defaults to False.

append_data(data, **kwargs)#

Append data. This is only possible if a datasource is specified and any kind of IOExtension or IOInterface is activated. By default, the builtin IOExtension is activated since version 0.1.12.

For the default interface, the datasource type and data type determine where the data will be stored and how the data has to look like. You can easily inherit from the IOExtension to customize read and write behaviour. If you import i.e. a timeseries to the same database as metacatalog, you will need to prepared data to to only hold an datetime index and the data to be stored.

New in version 0.1.12.

property checksum: str#

MD5 checksum of this entry. The checksum will change if any of the linked Metadata changes. This can be used in application built on metacatalog to verify integrity.

New in version 0.3.8.

create_datasource(path: str, type: Union[int, str], datatype: Union[int, str], variable_names: List[str] = None, commit: bool = False, **args) → DataSource#

Create a DataCource for this Entry. The data-source holds specific metadata about the actual type of source, the data resides in.

Parameters:

  • path (str) – The path to the data. This depends on the type of datasource used. This can be a URI, database table, local path etc.

  • type (metacatalog.models.DataSourceType) – type of the datasource

  • datatype (metacatalog.models.DataType) – Data Type of the referenced source

  • variable_names (list, optional) –

    New in version 0.9.1.

    List of variable names that store the data of the datasource of the entry. In tabular data, this is usually the column name(s) of the variable that is referenced by the Entry. In case of a netCDF file, this is the variable name(s) of the variable(s) that is/are referenced by the Entry. More generally, variable_names describes how a datasource would be indexed to retrieve the data of the entry.

  • commit (bool, optional) – If true, the created datasource will directly be commited to the database. If false (default) the model will be created and linked to this Entry, but you still need to add it to a database session and commit the session.

Returns:

datasource – Returns the newly created Datasource, which is also available as Entry.datasource.

Return type:

metacatalog.models.DataSource

delete_data(delete_source=False, **kwargs)#

Delete data. This is only possible if a datasource is specified and any kind of IOExtension or IOInterface is activated. By default, the builtin IOExtension is activated since version 0.1.12.

For the default interface, the datasource type and data type determine where the data is stored and how the data will be delted. You can easily inherit from the IOExtension to customize read and write behaviour.

New in version 0.1.12.

Parameters:

delete_source (bool) – If True, the DataSource will be deleted as well after the data has been deleted.

details_dict(full: bool = True) → dict#

Returns the associated details as dictionary.

Parameters:

full (bool) – If True (default) the keywords will contain the full info including key description, ids and stemmed key. If false, it will be truncated to a plain key:value dict

details_table(fmt: str = 'html') → str#

Return the associated details as table

Parameters:

fmt (string) –

Can be one of:

  • html to return a HTML table

  • latex to return LaTeX table

  • markdown to return Markdown table

export(path: Optional[str] = None, fmt: str = 'JSON', **kwargs)#

Export the Entry. Exports the data using a metacatalog extension. Refer to the note below to learn more about export extensions.

Parameters:

  • path (str) – If set, the export will be written into a file at the given location.

  • fmt (str) – Export format. Each export extension should at least support json and XML export.

  • **kwargs – Any other argument given will be passed down to the actual export function.

Notes

Uses any extension prefixed with ‘export-’ activated, by passing itself to the extension. If not format-specific extension is activated, the default ExportExtension will be used. A method of same name as fmt on the extension will be used. If such a method is not present, the ‘export’ method is used and the fmt attribute will be passed along. This can be used for format specific extensions. Refer to the notes about custom extensions to learn more about writing your own export extension.

Consider this example:

from metacatalog.ext import MetacatalogExtensionInterface
import json

class RawJSONExtension(MetacatalogExtensionInterface):
    @classmethod
    def init_extension(cls):
        pass

    @classmethod
    def json(cls, entry, path, **kwargs):
        # get the dict
        data = entry.to_dict(stringify=True)
        if path is None:
            return data
        else:
            with open(path, 'w') as f:
                json.dump(data, f, indent=kwargs.get('indent', 4))

You can activate and use it like:

>> from metacatalog import config >> config.load_extension(‘export’, RawJSONEXtension) >> entry.export(path=’testfile.json’, fmt=’json’, indent=2)

classmethod from_dict(session: Session, data: dict) → Entry#

Create a new Entry in the database from the given dict.

New in version 0.4.8.

Changed in version 0.7.4: PersonRoles other than ‘author’ and ‘coAuthor’ can be imported as well.

Parameters:

data (dict) – The dictionary to create the Entry from.

Returns:

entry – The newly created Entry.

Return type:

Entry

Notes

Currently, uploading data sources and data records is not supported.

get_data(**kwargs)#

Read the data. This is only possible if a datasource is specified and any kind of IOExtension or IOInterface is activated. By default, the builtin IOExtension is activated since version 0.1.12.

Changed in version 0.1.12.

import_data(data, **kwargs)#

Import data. This is only possible if a datasource is specified and any kind of IOExtension or IOInterface is activated. By default, the builtin IOExtension is activated since version 0.1.12.

For the default interface, the datasource type and data type determine where the data will be stored and how the data has to look like. You can easily inherit from the IOExtension to customize read and write behaviour. If you import i.e. a timeseries to the same database as metacatalog, you will need to prepared data to to only hold an datetime index and the data to be stored.

Changed in version 0.1.12.

io_interface#

alias of IOExtension

keyword_list() → List[str]#

List of tagged keywords associated to this instance. The keywords are related via the association table.

keywords_dict()#
make_composite(others: List['Entry'] = [], title: str = None, description: str = None, commit: bool = False) → EntryGroup#

Create a composite EntryGroup from this Entry. A composite marks stand-alone (is_partial = False) entries as inseparable. A composite can also contain a partial Entry (is_partial = True), whichs data only makes sense in the context of the composite group.

Parameters:

  • others (list of Entry) – The other Entries that should be part of the composite.

  • title (str) – Optional title of the composite, if applicable

  • description (str) – Optional description of the composite if applicable

  • commit (bool) – If True, the newly created Group will be persisted in the database. Defaults to False.

Returns:

composite – The newly created EntryGroup of EntryGroupType.name == ‘Composite’

Return type:

metacatalog.models.EntryGroup

neighbors(distance: Union[int, float], unit: str = 'meter', buffer_epsg: int = 3857, as_sql: bool = False, **kwargs) → List[Entry]#

Find neighboring Entries around the location of this instance. You can return the result, or the sqlalchemy Query object, which can be printed as plain SQL.

Parameters:

  • distance (int, float) – The maximum distance at which another Entry is still considered to be a neighbor.

  • unit (str) – Has to be one of [‘meter’, ‘km’, ‘mile’, ‘nautic’] to specify the unit of the given distance. Note that the distance will always be transformed into meter.

  • buffer_epsg (int) –

    The EPSG identification number of any projected cartesian coordinate reference system that uses meter as unit. This CRS will be used to apply the search distance (in meter).

    Note

    The default system is the transversal Mercartor projection, which is a global system. Thus, it can always be applied, but may introduce large uncertainties in small areas. Replace this attribute by a local CRS wherever possible.

  • as_sql (bool) – If False (default) the SQL query for neighbors will be executed and the result is returned. Else, the SQL query itself will be returned.

  • kwargs (keyword arguments) – Any passed keyword argument will be passed down to the api.find_entry function to further filter the results.

See also

around find_entry

plain_keywords_dict() → List[Dict[str, str]]#

Get a list of dictionaries containing a dict representation of each associated keyword to this Entry.

plain_keywords_list() → List[str]#

Returns list of controlled keywords associated with this instance of meta data. The List only includes the full path

set_new_author(new_author: Person, commit: bool = False)#

Set a new first Author for this entry.

Parameters:

  • new_author (metacatalog.models.Person) – The new first author. As of now the new author has to be passed as a model instance. Passing the ID or query parameter is not yet supported.

  • commit (boolean) –

    If True, the whole Entry will commit and persist itself to the database.

    Note

    This will also affect other uncommited edits to the Entry.

to_dict(deep: bool = False, stringify: bool = False) → dict#

Return the model as a python dictionary.

Changed in version 0.7.4: The dictionary now contains all persons roles

Parameters:

  • deep (bool) – If True, all related objects will be included as dictionary. Defaults to False

  • stringify (bool) – If True, all values will be turned into a string, to make the object serializable.

Returns:

obj – The Model as dict

Return type:

dict

class metacatalog.models.entrygroup.EntryGroup(**kwargs)#

An EntryGroup is an association object between any number of Entry records. The type of association is further described by EntryGroupType.

id#

Unique id of the record. If not specified, the database will assign it.

Type:

int

uuid#

New in version 0.1.9.

Version 4 UUID string to identify the Entry across installations. This field is read-only and will be assigned on creation. It is primarily used to export Entry into ISO19115 metadata.

Type:

str

type#

The type is most important to give meaning to a group of entries. Three types ship with metacatalog 'Project', 'Composite' and 'Split dataset'.

Type:

metacatalog.models.EntryGroupType

type_id#

Foreign key to the EntryGroupType

Type:

int

title#

A descriptive title for the Group. Maximum of 40 letters.

Type:

str

description#

An optional full text description of the EntryGroup. This description should contain all information necessary to understand the grouping.

Type:

str

publication#

New in version 0.1.9.

Autogenerated field giving the UTC timestamp of record creation.

Type:

datetime.datetime

lastUpdate#

Autogenerated field giving the UTC timestamp of last edit.

Type:

datetime.datetime

property checksum: str#

New in version 0.3.9.

MD5 checksum of this entry. The checksum will change if any of the linked Entries’ checksum changes. This can be used in application built on metacatalog to verify integrity.

export(path: Optional[str] = None, fmt: str = 'JSON', **kwargs)#

Export the EntryGroup. Exports the data using a metacatalog extension. Refer to the note below to learn more about export extensions.

Parameters:

  • path (str) – If set, the export will be written into a file at the given location.

  • fmt (str) – Export format. Each export extension should at least support json and XML export.

  • **kwargs – Any other argument given will be passed down to the actual export function.

Notes

Uses any extension prefixed with ‘export-’ activated, by passing itself to the extension. If not format-specific extension is activated, the default ExportExtension will be used. A method of same name as fmt on the extension will be used. If such a method is not present, the ‘export’ method is used and the fmt attribute will be passed along. This can be used for format specific extensions. Refer to the notes about custom extensions to learn more about writing your own export extension.

Consider this example:

from metacatalog.ext import MetacatalogExtensionInterface
import json

class RawJSONExtension(MetacatalogExtensionInterface):
    @classmethod
    def init_extension(cls):
        pass

    @classmethod
    def json(cls, group, path, **kwargs):
        # get the dict
        data = group.to_dict(stringify=True)
        if path is None:
            return data
        else:
            with open(path, 'w') as f:
                json.dump(data, f, indent=kwargs.get('indent', 4))

You can activate and use it like:

>> from metacatalog import ext >> ext.extension(‘export’, RawJSONEXtension) >> group.export(path=’testfile.json’, fmt=’json’, indent=2)

to_dict(deep: bool = False, stringify: bool = False) → dict#

To dict

Return the model as a python dictionary.

Changed in version 0.3.8: added stringify option

Parameters:

  • deep (bool) – If True, all related objects will be included as dictionary. Defaults to False

  • stringify (bool) – If True, all values will be turned into a string, to make the object serializable.

Returns:

obj – The Model as dict

Return type:

dict

class metacatalog.models.entrygroup.EntryGroupAssociation(**kwargs)#
class metacatalog.models.entrygroup.EntryGroupType(**kwargs)#
to_dict(deep: bool = False) → dict#

To dict

Return the model as a python dictionary.

Parameters:

deep (bool) – If True, all related objects will be included as dictionary. Defaults to False

Returns:

obj – The Model as dict

Return type:

dict

On this page
Show Source