estraces API documentation

eshard’s Trace set Python library.

esTraces provides the basic tools to handle side-channel traces in a unified way, through:
  • classes dedicated to trace and trace set manipulation: estraces.TraceHeaderSet, estraces.Trace

  • specific file formats reader: binaries formats and eShard’s format ETS

  • abstract base class to easily implement a reader for any specific file format

Trace and TraceHeaderSet

estraces.Trace and estraces.TraceHeaderSet are the classes which abstract your samples and the corresponding metadatas. Through these objects, you can: * manipulate your trace set with TraceHeaderSet - filter it, slice it, … * manipulate your samples data through the samples attribute of both TraceHeaderSet and Trace, which provides a Numpy array-like API. * manipulate your metadata through the metadatas attribute of both TraceHeaderSet and Trace, which provides a dict-like interface to your metadatas.

All the underlying mechanics to read datas on your trace set files are managed by a format reader.

Trace class

class estraces.Trace(trace_id, reader)[source]

Provide a consistent API to manipulate a trace samples data and metadatas.

samples

1 dimension samples data

Type:

Samples

metadatas

trace metadatas

Type:

Metadatas

headers

headers value of the trace set

Type:

Headers

Note

All metadatas are available through the metadatas attributes and through a corresponding named property.

Examples

Samples are available as 1 dimensional array-like Samples with shape (size of trace,). It supports a subset of numpy.ndarray-like slicing - including advanced list-based slicing:

trace.samples[2000:3000]
trace.samples[[1, 100, 1000]] # get samples at 1, 100 and 1000 indexes

Metadatas are available through a dict-like Metadatas instance:

metadatas = trace.metadatas
metadatas['plaintext']

Each metadata can be reached with its own property:

trace.plaintext # is equivalent to trace.metadatas['plaintext']

Headers are metadata value which are the same for all the traces of one given trace set. It is provided at the trace level through a dict-like object:

trace.headers[‘key’] # equivalent to ths.headers[‘key’] where ths is the trace header set of the trace.

TraceHeaderSet class

class estraces.TraceHeaderSet(reader, name='')[source]

Provide a consistent API for manipulating samples data and metadatas from any kind of trace files.

name

name of traces set

Type:

str

metadatas

dict-like metadatas object

Type:

Metadatas

headers

mapping of headers for this trace set, ie one off value metadata.

Type:

Headers

samples

2 dimensions samples object

Type:

Samples

traces

list of Trace instances

Type:

list

Notes

Each metadata available in metadatas attribute can be reached through a property.

Examples

With a TraceHeaderSet instance you can manipulate underlying Traces, samples data, metadatas, headers, slice, iterate or filter on your set.

Iterate on TraceHeaderSet instance:

for trace in ths:
    trace # each item is a :class:`Trace` instance

Slice is a new TraceHeaderSet instance limited to the slice:

new_ths = ths[:200]
new_ths = ths[[1, 2, 100]] # list-based slice supported

Get a Trace instance with an integer:

trace = ths[200] # get item returns a :class:`Trace` instance

You can also get the traces iterable:

ths.traces

Samples are available as 2 dimensional array-like Samples with shape (number of traces, size of traces). It supports a subset of numpy.ndarray-like slicing - including advanced list-based slicing:

ths.samples[2000:3000] # slice on traces, keeping all samples
ths.samples[:, 1000:2000] # slice on samples data, for all traces
ths.samples[[1, 100, 1000], :] # get samples for traces with indexes 1, 100 and 100

Metadatas are available through a dict-like Metadatas instance:

metadatas = ths.metadatas
metadatas['plaintext']

Each metadata can be reached with its own attribute:

ths.plaintext # is equivalent to ths.metadatas['plaintext']

Headers are metadata which have only one value for all the trace set. It is hold on a mapping, dict like object:

ths.headers[‘key’]

Filter traces in your ths with any arbitrary filtering function. As an example, get a new TraceHeaderSet with traces which first samples value is even:

filtered_ths = ths.filter(lambda trace: trace.samples[0] % 2 == 0)

Warning

TraceHeaderSet instance must be obtained through the use of a factory function suited to the use of your files format.

property traces

Provide a list of Trace instances from this trace header set.

property metadatas

Provide a Metadatas mapping-like object. Provides metadatas values for this trace header set.

property samples

Provide a 2d Samples instance ndarray-like of shape (number of traces, size of traces).

split(part_size)[source]

Return an iterable of TraceHeaderSet slices of at least length part_size.

Parameters:

part_size (int) – number of traces per slice

Returns:

(SplittedTraceHeaderSetIterable) An iterable TraceHeaderSet objects.

filter(filter_function)[source]

Build a new TraceHeaderSet instance based on traces passing the filter_function condition.

Parameters:

filter_function (callable) – function or lambda condition. Must take a trace as argument and return True or False.

Returns:

(TraceHeaderSet) a new instance with only selected traces.

Samples class

class estraces.Samples(reader, trace_id=None)[source]

Provide a wrapper object around Numpy ndarray for traces samples data manipulation.

A Samples instance can have 1 or 2 dimensions. It supports a reduced slicing and indexing API. ndarray API methods calls are proxied to the wrapped ndarray.

array

wrapped Numpy array.

Type:

ndarray

Metadatas class

class estraces.Metadatas(reader, trace_id=None)[source]

Provides a dict-like object of traces metadatas.

Each metadata value is either a value, if the Metadata instance wraps one trace, or an array of values, if the Metadata wraps a trace set metadatas.

Get a trace header set from files - Format readers

Basically, formats reader are classes which implements methods to read samples and metadatas from a trace set files. For the sake of simplicity, a format should always comes with a simple factory function to get a ths from basic parameters. esTraces provides:

Bin reader format: read_ths_from_bin_… functions

estraces.read_ths_from_bin_filenames_pattern(filename_pattern, metadatas_parsers, dtype, headers={}, offset=0, padding_mode=PaddingMode.NONE)[source]

Build and return a TraceHeaderSet instance from binaries files following a filename pattern.

Each file must contains samples data for one trace.

Parameters:
  • filename_pattern (str) – binary files filename pattern, for example "samples/*.bin"

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a bin_extractor.HeaderExtractor, bin_extractor.PatternExtractor, bin_extractor.FilePatternExtractor or bin_extractor.DirectValue instance.

  • dtype (dtype) – the Numpy samples data dtype

  • headers (dict, default={}) – dictionary containing headers values for this trace set.

  • (int (offset) – 0): use as an offset when reading samples in files.

  • default – 0): use as an offset when reading samples in files.

:param padding_mode (PaddingMode: PaddingMode.NONE): choose how to handle different traces size in your trace list.

Possible modes are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Parameters:

defaultPaddingMode.NONE): choose how to handle different traces size in your trace list. Possible modes are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Returns:

(TraceHeaderSet)

estraces.read_ths_from_bin_filenames_list(filenames_list, metadatas_parsers, dtype, headers={}, offset=0, padding_mode=PaddingMode.NONE)[source]

Build and return a TraceHeaderSet instance from binaries files listed.

Each file must contains samples data for one trace.

Parameters:
  • filenames_list (list) – binary files filenames list

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a bin_extractor.HeaderExtractor, bin_extractor.PatternExtractor, bin_extractor.FilePatternExtractor or bin_extractor.DirectValue instance.

  • dtype (dtype) – the Numpy samples data dtype

  • headers (dict, default={}) – dictionary containing headers values for this trace set.

  • (int (offset) – 0): use as an offset when reading samples in files.

  • default – 0): use as an offset when reading samples in files.

:param padding_mode (PaddingMode: PaddingMode.NONE): choose how to handle different traces size in your trace list.

Possible values are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Parameters:

defaultPaddingMode.NONE): choose how to handle different traces size in your trace list. Possible values are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Returns:

(TraceHeaderSet)

class estraces.formats.bin_extractor.DirectValue(value=None, unhexlify=True)[source]

Object used to handle user defined values for metadata.

value

the direct value to set as a metadata

unhexlify

get the binary data out of the hexadecimal representation of value (default: True)

When unhexlify is True (default), the expected inputs are either a hexadecimal string without the ‘0x’ prefix or an integer list in range(0, 256). When unhexlify is False, it can be any regular string.

Returns:

a DirectValue() object

Usage example:

d0 = DirectValue(“00AA11BB”) d1 = DirectValue([0, 170, 17, 187]) d2 = DirectValue(“John Dow”, unhexlify=False)

get_text(_)[source]

Get the user input to be used as Direct Value.

Returns:

byte array

class estraces.formats.bin_extractor.FilePatternExtractor(filename, *args, **kwargs)[source]

Object iterating PatternExtractor to parse files.

class estraces.formats.bin_extractor.HeaderExtractor(start=None, end=None, count=None, unhexlify=True)[source]

Object to retrieve metadata from binary files.

The user must specify a start reading offset, and either the end offset or the number of bytes to read.

start

start reading offset

end

end reading offset

count

the number of byte to read

Usage:

plain = HeaderExtractor(start=0, end=15) cipher = HeaderExtractor(start=16, count=16)

get_text(filename)[source]

Get the text from a binary hexadecimal string contained in a file header.

get_text_stacked(header)[source]

Get the text from a binary header contained in a stacked file.

class estraces.formats.bin_extractor.PatternExtractor(pattern=None, replace=None, num=0, unhexlify=True)[source]

Object using regular expressions to parse strings.

pattern

a string specifying the regular expression.

replace

the replacement pattern using regex groups.

unhexlify

returns the binary data represented by the hexadecimal string value.

get_pattern(string)[source]

Return the matching PatternExtractor.

The pattern that matches the regex self.pattern in the input string will be returned. According to self.unhexlify, it will return a string or binary data.

ETS reader format: read_ths_from_ets_file

estraces.read_ths_from_ets_file(filename)[source]

Build and return a TraceHeaderSet instance from an ETS file.

Parameters:

filename (str or Path) – filename of the ETS file

Returns:

(TraceHeaderSet)

TRS reader format: read_ths_from_trs_file

estraces.read_ths_from_trs_file(filename, metadatas_parsers, dtype=None)[source]

Build and return a TraceHeaderSet instance from TraceRiscureSet file.

Parameters:
  • filename (str) – TRS filename

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a function or lambda taking data as input and returning the metadata value parsed from it. For TRS V2 files, the metadata_parsers argument can be None. In this case, it is built with the TRACE_PARAMETER_DEFINITIONS header.

  • dtype (dtype, optional) – a Numpy dtype that will override the TRS samples data type

Returns:

(TraceHeaderSet)

Get a trace header set from numpy arrays

The read_ths_from_ram function allows to build a TraceHeaderSet from any arrays, in memory.

RAM reader format: read_ths_from_ram function

estraces.read_ths_from_ram(samples, **kwargs)[source]

Build and return a TraceHeaderSet instance from arrays in RAM memory.

Parameters:
  • samples (ndarray) – traces samples

  • **kwargs (ndarray) – metadata as ndarray

Returns:

(TraceHeaderSet)

Get a trace header set concatenated from multiple trace header set instances

The read_ths_from_multiple_ths function allows to build a TraceHeaderSet from several others TraceHeaderSet instance, as a concatenation.

estraces.read_ths_from_multiple_ths(*args, **kwargs)[source]

Build and return a TraceHeaderSet instance as the concatenation of all input THSs.

Parameters:
  • args – some TraceHeaderSet objects.

  • kwargs – some TraceHeaderSet objects.

You can pass a headers dict argument to override inconsistent headers between trace header sets or add custom values.

Note

shape and metadata must be consistent between each ths.

Returns:

(TraceHeaderSet)

Writing an ETS file

The class ETSWriter provides API to create Eshard Trace Set file.

class estraces.ETSWriter(filename, overwrite=False)[source]

Provide API to create an ETS (Eshard Trace Set) file.

Implementing a new format reader

To handle your own trace set format, you must create a format reader class inheriting from estraces.AbstractReader. A reader implementation must basically implement two methods: * a fetch_samples method which takes a traces id list and samples frame as inputs, and returns a 2 dimension numpy array * a fetch_metadatas method which takes a trace id and a metadata name, and returns the metadata values

When developing your own format reader, you should also provide a factory function to get a TraceHeaderSet instance from the files.

The base format reader AbstractReader

class estraces.AbstractReader[source]

Provides the basic interface any format reader for trace set must provides.

The TraceHeaderSet and Trace abstraction rely on the use of concrete implementation of AbstractReader. Format implementation must provides fetching methods for samples and metadatas, and must implement __getitem__. All performances strategies (lazy loading, caching, memory or CPU usage) are thus delegated to the implementer of the concrete format.

fetch_samples()[source]

given traces and samples frame, returns samples data

fetch_metadatas()[source]

given a trace id and a metadata key, returns metadata values

__getitem__()[source]

handles slicing of reader

metadatas_keys()

property - Returns a view or container of the metadatas available keys

abstract fetch_samples(traces, frame=None)[source]

Fetch samples for the given traces id and given samples data frame.

Parameters:
  • traces (list) – Lists of traces id to fetch.

  • frame – Samples data to fetch. Must support Ellipsis, slice, list, ndarray or int types.

Return type:

ndarray

Returns:

(numpy.ndarray) array of shape (number of traces, size of samples)

abstract fetch_metadatas(key, trace_id=None)[source]

Fetch metadata value for the given metadata key and trace id.

Parameters:
  • key (Hashable) – Key of the metadata to fetch. Must be hashable.

  • trace_id (int) – Trace id for which to fetch the metadata.

Return type:

Union[TypeVar(M), Container[TypeVar(M)]]

Returns:

A container of all the values of the trace set for the given metadata if trace_id is None. Else, the value of the metadata for the given trace id.

abstract fetch_header(key)[source]

Fetch header value for the given key.

Parameters:

key (Hashable) – key of the header to fetch.

Returns:

the header value.

abstract property metadatas_keys

Provides a list or views of the metadatas keys available.

abstract get_trace_size(trace_id)[source]

Provides the size of trace trace_id.

abstract property headers_keys

Provides a list or view of the headers keys available.

How to create a TraceHeaderSet instance

estraces.build_trace_header_set(reader, name)[source]

Factory function which provides easy instantiation of TraceHeaderSet instance.

The function first creates a class, inheriting from TraceHeaderSet, suited to the format instance provided, instantiate a trace header set and returns it.

Warning

End user should not use directly this function, but factory functions suited to their concrete traces file format, provided by format developers.

Parameters:
Raises:

TypeError – if reader does not implement AbstractReader.

Returns:

trace header set instance.

Return type:

TraceHeaderSet

copyright:
  1. 2022 ESHARD


class estraces.AbstractReader[source]

Bases: ABC

Provides the basic interface any format reader for trace set must provides.

The TraceHeaderSet and Trace abstraction rely on the use of concrete implementation of AbstractReader. Format implementation must provides fetching methods for samples and metadatas, and must implement __getitem__. All performances strategies (lazy loading, caching, memory or CPU usage) are thus delegated to the implementer of the concrete format.

fetch_samples()[source]

given traces and samples frame, returns samples data

fetch_metadatas()[source]

given a trace id and a metadata key, returns metadata values

__getitem__()[source]

handles slicing of reader

metadatas_keys()

property - Returns a view or container of the metadatas available keys

abstract fetch_header(key)[source]

Fetch header value for the given key.

Parameters:

key (Hashable) – key of the header to fetch.

Returns:

the header value.

abstract fetch_metadatas(key, trace_id=None)[source]

Fetch metadata value for the given metadata key and trace id.

Parameters:
  • key (Hashable) – Key of the metadata to fetch. Must be hashable.

  • trace_id (int) – Trace id for which to fetch the metadata.

Return type:

Union[TypeVar(M), Container[TypeVar(M)]]

Returns:

A container of all the values of the trace set for the given metadata if trace_id is None. Else, the value of the metadata for the given trace id.

abstract fetch_samples(traces, frame=None)[source]

Fetch samples for the given traces id and given samples data frame.

Parameters:
  • traces (list) – Lists of traces id to fetch.

  • frame – Samples data to fetch. Must support Ellipsis, slice, list, ndarray or int types.

Return type:

ndarray

Returns:

(numpy.ndarray) array of shape (number of traces, size of samples)

abstract get_trace_size(trace_id)[source]

Provides the size of trace trace_id.

abstract property headers_keys

Provides a list or view of the headers keys available.

abstract property metadatas_keys

Provides a list or views of the metadatas keys available.

class estraces.ETSWriter(filename, overwrite=False)[source]

Bases: object

Provide API to create an ETS (Eshard Trace Set) file.

add_metadata(metadata, batch_size=1)[source]

Append provided metadata to the end of ETS.

Warning: when using directly this method, it is up to you to ensure consistency of indices between samples and metadata in your ETS file. Use of add_trace or add_trace_header_set methods is recommended.

Parameters:

metadata (Metadatas) – a metadata instance.

add_samples(samples, batch_size=1)[source]

Append provided samples to the end of ETS.

Warning: when using directly this method, it is up to you to ensure consistency of indices between samples and metadata in your ETS file. Use of add_trace or add_trace_header_set methods is recommended.

Parameters:

samples (Samples) – a samples instance.

add_trace(trace)[source]

Append trace samples and metadata provided to the end of the ETS.

Parameters:

trace (Trace) – a Trace instance.

add_trace_header_set(trace_header_set)[source]

Append all traces samples and metadata hold by trace_header_set at the end of the ETS.

Parameters:

trace_header_set (TraceHeaderSet) – a trace header set instance.

close()[source]

Close the current instance ETS file.

get_reader()[source]

Returns a TraceHeaderSet instance from the current writer, and closes it.

write_headers(headers)[source]
write_meta(tag, metadata, index=None)[source]
write_metadata(key, value, index=None, scalar=False)[source]

Write provided metadata at specified index of the ETS.

If index is None, data will be appended to the end of the ETS.

Warning: when using directly this method, it is up to you to ensure consistency of indices between samples and metadata in your ETS file.

Parameters:
  • key (str) – metadata key

  • value (numpy.ndarray) – metadata value to write.

  • index (int, default=None) – index at which to write.

write_points(points, index=None)[source]
write_samples(samples_array, index=None)[source]

Write provided trace samples at specified index of the ETS.

If index is None, data will be appended to the end of the ETS.

Warning: when using directly this method, it is up to you to ensure consistency of indices between samples and metadata in your ETS file.

Parameters:
  • samples_array (numpy.ndarray) – samples to write.

  • index (int, default=None) – index at which to write.

write_trace_object_and_points(trace_object, points, index=None)[source]

Write provided trace samples and metadata at specified index of the ETS.

If index is None, data will be appended to the end of the ETS.

Parameters:
  • trace_object (Trace) – a Trace instance.

  • points (numpy.ndarray) – samples to write.

  • index (int, default=None) – index at which to write.

exception estraces.ETSWriterError[source]

Bases: Exception

class estraces.Metadatas(reader, trace_id=None)[source]

Bases: Mapping

Provides a dict-like object of traces metadatas.

Each metadata value is either a value, if the Metadata instance wraps one trace, or an array of values, if the Metadata wraps a trace set metadatas.

is_trace()[source]
class estraces.PaddingMode(value)[source]

Bases: Enum

Define the padding mode used when traces of binary format have different length.

Possible modes are:
  • NONE: no padding is applied, an exception is thrown when instantiating the trace set.

  • PAD: samples will be padded with zero to the length of the longer trace of the set.

  • TRUNCATE: samples will be truncated to the length of the smaller trace of the set.

NONE = 0
PAD = 1
TRUNCATE = 2
class estraces.Samples(reader, trace_id=None)[source]

Bases: object

Provide a wrapper object around Numpy ndarray for traces samples data manipulation.

A Samples instance can have 1 or 2 dimensions. It supports a reduced slicing and indexing API. ndarray API methods calls are proxied to the wrapped ndarray.

array

wrapped Numpy array.

Type:

ndarray

SUPPORTED_INDICES_TYPES = (<class 'ellipsis'>, <class 'slice'>, <class 'int'>, <class 'list'>, <class 'numpy.ndarray'>, <class 'range'>)
property array
property dtype
property shape
property size
class estraces.Trace(trace_id, reader)[source]

Bases: object

Provide a consistent API to manipulate a trace samples data and metadatas.

samples

1 dimension samples data

Type:

Samples

metadatas

trace metadatas

Type:

Metadatas

headers

headers value of the trace set

Type:

Headers

Note

All metadatas are available through the metadatas attributes and through a corresponding named property.

Examples

Samples are available as 1 dimensional array-like Samples with shape (size of trace,). It supports a subset of numpy.ndarray-like slicing - including advanced list-based slicing:

trace.samples[2000:3000]
trace.samples[[1, 100, 1000]] # get samples at 1, 100 and 1000 indexes

Metadatas are available through a dict-like Metadatas instance:

metadatas = trace.metadatas
metadatas['plaintext']

Each metadata can be reached with its own property:

trace.plaintext # is equivalent to trace.metadatas['plaintext']

Headers are metadata value which are the same for all the traces of one given trace set. It is provided at the trace level through a dict-like object:

trace.headers[‘key’] # equivalent to ths.headers[‘key’] where ths is the trace header set of the trace.

property headers
property metadatas
property samples
class estraces.TraceHeaderSet(reader, name='')[source]

Bases: object

Provide a consistent API for manipulating samples data and metadatas from any kind of trace files.

name

name of traces set

Type:

str

metadatas

dict-like metadatas object

Type:

Metadatas

headers

mapping of headers for this trace set, ie one off value metadata.

Type:

Headers

samples

2 dimensions samples object

Type:

Samples

traces

list of Trace instances

Type:

list

Notes

Each metadata available in metadatas attribute can be reached through a property.

Examples

With a TraceHeaderSet instance you can manipulate underlying Traces, samples data, metadatas, headers, slice, iterate or filter on your set.

Iterate on TraceHeaderSet instance:

for trace in ths:
    trace # each item is a :class:`Trace` instance

Slice is a new TraceHeaderSet instance limited to the slice:

new_ths = ths[:200]
new_ths = ths[[1, 2, 100]] # list-based slice supported

Get a Trace instance with an integer:

trace = ths[200] # get item returns a :class:`Trace` instance

You can also get the traces iterable:

ths.traces

Samples are available as 2 dimensional array-like Samples with shape (number of traces, size of traces). It supports a subset of numpy.ndarray-like slicing - including advanced list-based slicing:

ths.samples[2000:3000] # slice on traces, keeping all samples
ths.samples[:, 1000:2000] # slice on samples data, for all traces
ths.samples[[1, 100, 1000], :] # get samples for traces with indexes 1, 100 and 100

Metadatas are available through a dict-like Metadatas instance:

metadatas = ths.metadatas
metadatas['plaintext']

Each metadata can be reached with its own attribute:

ths.plaintext # is equivalent to ths.metadatas['plaintext']

Headers are metadata which have only one value for all the trace set. It is hold on a mapping, dict like object:

ths.headers[‘key’]

Filter traces in your ths with any arbitrary filtering function. As an example, get a new TraceHeaderSet with traces which first samples value is even:

filtered_ths = ths.filter(lambda trace: trace.samples[0] % 2 == 0)

Warning

TraceHeaderSet instance must be obtained through the use of a factory function suited to the use of your files format.

filter(filter_function)[source]

Build a new TraceHeaderSet instance based on traces passing the filter_function condition.

Parameters:

filter_function (callable) – function or lambda condition. Must take a trace as argument and return True or False.

Returns:

(TraceHeaderSet) a new instance with only selected traces.

property headers
property metadatas

Provide a Metadatas mapping-like object. Provides metadatas values for this trace header set.

property samples

Provide a 2d Samples instance ndarray-like of shape (number of traces, size of traces).

split(part_size)[source]

Return an iterable of TraceHeaderSet slices of at least length part_size.

Parameters:

part_size (int) – number of traces per slice

Returns:

(SplittedTraceHeaderSetIterable) An iterable TraceHeaderSet objects.

property traces

Provide a list of Trace instances from this trace header set.

estraces.build_trace_header_set(reader, name)[source]

Factory function which provides easy instantiation of TraceHeaderSet instance.

The function first creates a class, inheriting from TraceHeaderSet, suited to the format instance provided, instantiate a trace header set and returns it.

Warning

End user should not use directly this function, but factory functions suited to their concrete traces file format, provided by format developers.

Parameters:
Raises:

TypeError – if reader does not implement AbstractReader.

Returns:

trace header set instance.

Return type:

TraceHeaderSet

estraces.compress_ets(filename, out_filename)[source]

Create a compressed version of an ETS file.

Creates a new ETS file using compression options provided by h5. The gain of compression can highly vary, depending on the nature of the data, from 10% to 50%. The trade-off is a somewhat slower reading from the resulting file.

Parameters:
  • filename (str) – filename of original uncompressed ETS file.

  • out_filename (str) – filename of output compressed ETS file.

Raises:
  • ValueError if a file named out_filename already exists.

  • AttributeError if the original file filename doesn't exists.

estraces.read_ths_from_bin_filenames_list(filenames_list, metadatas_parsers, dtype, headers={}, offset=0, padding_mode=PaddingMode.NONE)[source]

Build and return a TraceHeaderSet instance from binaries files listed.

Each file must contains samples data for one trace.

Parameters:
  • filenames_list (list) – binary files filenames list

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a bin_extractor.HeaderExtractor, bin_extractor.PatternExtractor, bin_extractor.FilePatternExtractor or bin_extractor.DirectValue instance.

  • dtype (dtype) – the Numpy samples data dtype

  • headers (dict, default={}) – dictionary containing headers values for this trace set.

  • (int (offset) – 0): use as an offset when reading samples in files.

  • default – 0): use as an offset when reading samples in files.

:param padding_mode (PaddingMode: PaddingMode.NONE): choose how to handle different traces size in your trace list.

Possible values are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Parameters:

defaultPaddingMode.NONE): choose how to handle different traces size in your trace list. Possible values are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Returns:

(TraceHeaderSet)

estraces.read_ths_from_bin_filenames_pattern(filename_pattern, metadatas_parsers, dtype, headers={}, offset=0, padding_mode=PaddingMode.NONE)[source]

Build and return a TraceHeaderSet instance from binaries files following a filename pattern.

Each file must contains samples data for one trace.

Parameters:
  • filename_pattern (str) – binary files filename pattern, for example "samples/*.bin"

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a bin_extractor.HeaderExtractor, bin_extractor.PatternExtractor, bin_extractor.FilePatternExtractor or bin_extractor.DirectValue instance.

  • dtype (dtype) – the Numpy samples data dtype

  • headers (dict, default={}) – dictionary containing headers values for this trace set.

  • (int (offset) – 0): use as an offset when reading samples in files.

  • default – 0): use as an offset when reading samples in files.

:param padding_mode (PaddingMode: PaddingMode.NONE): choose how to handle different traces size in your trace list.

Possible modes are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Parameters:

defaultPaddingMode.NONE): choose how to handle different traces size in your trace list. Possible modes are NONE, PAD and TRUNCATE (see bin_format.PaddingMode).

Returns:

(TraceHeaderSet)

estraces.read_ths_from_ets_file(filename)[source]

Build and return a TraceHeaderSet instance from an ETS file.

Parameters:

filename (str or Path) – filename of the ETS file

Returns:

(TraceHeaderSet)

estraces.read_ths_from_multiple_ths(*args, **kwargs)[source]

Build and return a TraceHeaderSet instance as the concatenation of all input THSs.

Parameters:
  • args – some TraceHeaderSet objects.

  • kwargs – some TraceHeaderSet objects.

You can pass a headers dict argument to override inconsistent headers between trace header sets or add custom values.

Note

shape and metadata must be consistent between each ths.

Returns:

(TraceHeaderSet)

estraces.read_ths_from_ram(samples, **kwargs)[source]

Build and return a TraceHeaderSet instance from arrays in RAM memory.

Parameters:
  • samples (ndarray) – traces samples

  • **kwargs (ndarray) – metadata as ndarray

Returns:

(TraceHeaderSet)

estraces.read_ths_from_sqlite(filename, samples_config, metadata_config=MetadataConfig(metadata_defs={}, table_name=None, pk_column_name='trace_id'), headers_config=HeadersConfig(table_name=None, pk_column_name='id', tag_column_name='tag', value_column_name='value', dtype_column_name='dtype'))[source]

Build and return a TraceHeaderSet instance read from a SQLite3 database file.

Parameters:
  • filename (str or Path) – filename of the SQLite3 database.

  • samples_config (SamplesConfig) – instance of SamplesConfig with parameters to read samples - table name, pk column name, samples column name and dtype format of samples data.

  • metadata_config (MetadataConfig) – instance of MetadataConfig with parameters to read trace metadata - table name, pk column name and dict where each entry is the column name of a metadata and the corresponding value the dtype of the metadata.

  • headers_config (HeadersConfig) – instance of HeadersConfig with parameters to read headers from a specific table - table name, tag column name, value column name and dtype column name.

Returns:

(TraceHeaderSet)

estraces.read_ths_from_trs_file(filename, metadatas_parsers, dtype=None)[source]

Build and return a TraceHeaderSet instance from TraceRiscureSet file.

Parameters:
  • filename (str) – TRS filename

  • metadatas_parsers (dict) – dict with a key for each metadata of the trace set, with value a function or lambda taking data as input and returning the metadata value parsed from it. For TRS V2 files, the metadata_parsers argument can be None. In this case, it is built with the TRACE_PARAMETER_DEFINITIONS header.

  • dtype (dtype, optional) – a Numpy dtype that will override the TRS samples data type

Returns:

(TraceHeaderSet)