Overview

The Buffer class can be used to open measurement data files that are generated by the Analyzer4D software. It can be used to access the Metadata of those files like process number, channel etc. and the actual measurement data. It can be used together with the BufferMetadataCache to organize large amounts of buffer files.

Example

In the example we provide the path to a buffer file to the Buffer class and use the with-statement to open it to read the process number.

1 from qass.tools.analyzer.buffer_parser import Buffer
2
3 buffer_file = "path/to/my/buffer_file"
4 with Buffer(buffer_file) as buff:
5    print(buff.process)

Buffer

The Buffer can be used to open measurement files created by the analyzer4D software

class qass.tools.analyzer.buffer_parser.Buffer(filepath)[source]
class ADCTYPE(value)[source]

An enumeration.

class DATAKIND(value)[source]

An enumeration.

class DATAMODE(value)[source]

An enumeration.

class DATATYPE(value)[source]

An enumeration.

property adc_type

The type of analog/digital converter. The types are defined in class ADC_TYPE with the most important being ADC_16BIT and ADC_24BIT. If you want to obtain the name of the constant, use adc_type.name.

Returns:

The ADCTYPE constant

Return type:

enum class reference ADCTYPE

property analyzer_version: str | None

The analyzer4D version as a dot separated string consisting of major.minor.micro.patch :rtype: str, None

property avg_frq

Property which returns the moving average factor along the frequency axis.

Returns:

Factor of the moving average.

Return type:

int

property avg_time

Property which returns the moving average factor along the time axis.

Returns:

Factor of the moving average.

Return type:

int

property bit_resolution

this is not the type of analog digital converter used. The usual value is 16. If no logarithm is applied to the data the value is 32.

Returns:

The bit resolution.

Return type:

int

Type:

The bit resolution after the FFT. Caution

block_infos(columns: List[str] = ['preamp_gain', 'mux_port', 'measure_positions', 'inputs', 'outputs'], changes_only: bool = False, fast_jump: bool = True)[source]

block_infos iterates through all memory blocks of a buffer file (typically one MB) and fetches the subdata information each memory block has one set of metadata but e.g. 65 subdata entries for raw files or more than 2000 entries for a 32 times compressed file, where each entry contains meta information about a small time frame (15 spectrums in raw fft files)

Parameters:

columns – List of columns that should be in the result_array.

You can define the order of the columns here. Possible column names are: [‘preamp_gain’, ‘mux_port’, ‘measure_positions’, ‘inputs’, ‘outputs’, ‘times’, ‘index’, ‘spectrums’] :type columns: List[str]

Parameters:

changes_only – Flag to enable a summarized output. If True the output does not contain all entries but only entries where the data columns changed.

In the current implementation the memory consumption does not change here -> in both cases the array is first completely built. Defaults to False. :type changes_only: bool

Parameters:

fast_jump – If True the function uses seeking in the file instead of parsing every single datablock header.

The offsets are investigated for the first datablock header and simply applied for all other datablock headers. Seeking is usually faster than parsing the datablock headers if they are not already in the cache. If the datablock headers are already cached this flag has no effect. Defaults to true. :type fast_jump: bool

Returns:

header_infos, an array containing (spec_index), (index), (times), preamp_gain, mux_port, measure_position, 24bit input, 16bit output, (times), (index), (spec_index)

Return type:

numpy array of int64, if times are included otherwise int32

property bytes_per_sample

Number of bytes per sample.

Returns:

Number of bytes per sample (usually 2 bytes).

Return type:

int

property channel

The channel number used.

Returns:

The channel number.

Return type:

int

property compression_frq

Property which returns the frequency compression factor.

Returns:

Frequency compression factor.

Return type:

int

property compression_time

Property which returns the time compression factor.

Returns:

Time compression factor.

Return type:

int

property datakind

The data kind constant is a constant which provides additional buffer specifications. A common value often is KIND_UNDEF. If you want to obtain the name of the constant, use datakind.name.

Returns:

The data kind constant.

Return type:

enum class defined in DATAKIND

property datamode

The data mode is a constant which specifies what kind of data are in the buffer. The most important ones are DATAMODE_FFT and DATAMODE_SIGNAL. If you want to obtain the name of the constant, use datamode.name.

Returns:

The data mode constant

Return type:

enum class defined in DATAMODE

property datatype

The data type constant is a constant which specifies different compression and buffer types. The most important one being COMP_RAW. If you want to obtain the name of the constant, use datatype.name.

Returns:

The data type constant.

Return type:

enum class defined in DATATYPE

property db_count

Returns the number of data blocks.

Returns:

Number of data blocks.

:rtype int

db_header(db_idx)[source]

The data block header contains key words and their corresponding values. This information is retrieved and stored in a dictionary. The values can be accessed by providing the corresponding key. Please ensure that the key exists in the database prior to its use or use the get method to access its content.Otherwise this will cause a runtime error.

Parameters:

db_idx (int) – data block index

Raises:

Value_Error – The data block index is out of range.

Returns:

a dictionary with the keywords and values

Return type:

dictionary

property db_header_size

Each data block has a header. This property provides the size of the data block header in bytes. The sizes of all data block headers are equal.

Returns:

Size in bytes

Return type:

int

db_header_spec(spec: int)[source]

The method returns the data block header as a dictionary of the data block containing the spectrum whose index is provided.

Parameters:

spec (int) – The index of a spectrum.

Returns:

The data block header with keywords and values.

Return type:

dictionary

property db_sample_count

Returns the number of samples in a completely filled data block. It is calculated by dividing the data block size by the number of bytes per sample.

Returns:

Number of samples.

Return type:

int

property db_size

Returns the size of a completely filled data block in bytes.

Returns:

Size of a completely filled data block.

Return type:

int

property db_spec_count

The number of spectra within a filled data block. It is calculated by dividing the number of samples in a data block by the number of frequency bands.

Returns:

Number of spectra in a filled data block.

Return type:

int

db_value(db_idx: int, key: str)[source]

Similar to the file header each data block has its own header. This method returns the value of the property specified by key in the specified data block. The first data block has the index 0.

Parameters:

  • db_idx (int) – The index of the data block.

  • key (string) – The string containing the key word.

Returns:

The data block value for the given key.

Return type:

int

static delog(data_arr, fft_log_shift, ad_bit_resolution)[source]

Method to de-logarithmize a data array. Usually when data are measured with the Optimizer the data are logarithmized. The problem with logarithms is that adding them up constitutes a multiplication of the non logarithmized data. So in order to carry out calculations such as calculating sums etc. it is crucial to use de-logarithmized data.

Parameters:

  • data_arr (numpy ndarray) – Array with the logarithmized data.

  • fft_log_shift (int) – Base of the logarithm.

  • ad_bit_resolution (int) – The bit resolution (usually 16).

Returns:

Array with de-logarithmized data.

Return type:

numpy ndarray of floats

property fft_log_shift

Base of the logarithm applied to the data. Please note that a 1 needs to be added to the value.

Returns:

Base of the logarithm.

Return type:

int

file_size()[source]

The file size of the buffer file. The value is not stored in the header but retrieved using the operating system.

Returns:

file size in bytes

Return type:

int

property filepath

Returns the complete path to the file.

Returns:

Path to the file

Return type:

string

property frq_bands

Each spectrum has a maximum sample number of 512. This is the maximum number of frequency bands. This figure decreases with compression along the frequency axis. The corresponding key word in the metainfo dictionary ‘s_p_fram’ (samples per frame).

Returns:

Number of frequency bands.

Return type:

int

property frq_end

The property returns the highest represented frequency.

Returns:

Frequency in Hertz (Hz).

Return type:

int

property frq_per_band

The property returns the frequency range per band. The maximum number of bands is 512.

Returns:

Frequency range in Hertz (Hz) for one band.

Return type:

float

property frq_start

The property returns the lowest represented frequency.

Returns:

Frequency in Hertz (Hz).

Return type:

int

property full_blocks

All but the last data block are usually completly filled. In order to calculate the number of those data blocks which are completely filled the header size of the file is substracted from the file size. This figure is divided by the sum of the data block header size and data block size and rounded down to the nearest integer.

Returns:

Number of full data blocks.

Return type:

int

getArray(specFrom=None, specTo=None, delog=None)[source]

Wrapper function to ‘get_data’. This function provides access to the measurement data in the buffer file. The data are retrieved for the range of spectra and stored in a numpy array. The data can be logarithmized or not. The datatype of the array depends on the type of buffer.

Parameters:

  • specFrom (int, optional) – First spectrum to be retrieved (default first available spectrum).

  • specTo (int, optional) – Last spectrum to be retrieved (default last available spectrum).

  • delog (boolean, optional) – To de-logarithmize the data (default None).

Raises:

  • InvalidArgumentError – The specFrom value is out of range.

  • InvalidArgumentError – The specTo value is out of range.

Returns:

The buffers data.

Return type:

numpy ndarray

 1import qass.tools.analyzer.buffer_parser as bp
 2proc = range(1,100,1)
 3# path contains the path to a directory containing buffer files
 4buff = bp.filter_buffers(path, {'wanted_process': proc , 'datamode': bp.Buffer.DATAMODE.DATAMODE_FFT})
 5for buffer in buff:
 6    buff_file = (buffer.filepath)
 7
 8with bp.Buffer(buff_file) as buff:
 9    spec_start = 0
10    spec_end = (buff.db_count -1) * buff.db_spec_count
11    print('Spec_end: ' + str(spec_end))
12    data = buff.get_array(spec_start, spec_end, True)
getRealSpecCount()[source]

Wrapper function for the property spec_count. The number of spectra within a filled data block. It is calculated by dividing the number of samples in a data block by the number of frequency bands.

Returns:

Number of spectra in a filled data block.

Return type:

int

getSpecDuration()[source]

Wrapper Function for property spec_duration.

get_data(specFrom=None, specTo=None, conversion: str | None = None)[source]

This function provides access to the measurement data in the buffer file. The data are retrieved for the range of spectra and stored in a numpy array. The data can be logarithmized or not. The datatype of the array depends on the type of buffer.

Parameters:

  • specFrom (int, optional) – First spectrum to be retrieved (default first available spectrum)

  • specTo (int, optional) – Last spectrum to be retrieved (default last available spectrum)

  • conversion (string, optional) – Conversion (‘log’ or ‘delog’) of the data.

Raises:

  • InvalidArgumentError – The specFrom value is out of range

  • InvalidArgumentError – The specTo value is out of range

  • ValueError – The given indices exceed the buffer file’s size

  • InvalidArgumentError – The given conversion is unknown

Returns:

The buffers data.

Return type:

numpy ndarray

 1import qass.tools.analyzer.buffer_parser as bp
 2proc = range(1,100,1)
 3# path contains the path to a directory containing buffer files
 4buff = bp.filter_buffers(path, {'wanted_process': proc , 'datamode': bp.Buffer.DATAMODE.DATAMODE_FFT})
 5for buffer in buff:
 6    buff_file = (buffer.filepath)
 7
 8with bp.Buffer(buff_file) as buff:
 9    spec_start = 0
10    spec_end = (buff.db_count -1) * buff.db_spec_count
11    print('Spec_end: ' + str(spec_end))
12    conv = 'delog'
13    data = buff.get_data(spec_start, spec_end, conv)
property header_hash

A hash value for the buffer’s header. The header of the buffer is almost unique for a buffer. A buffer should only differ in the length of the contained data.

Returns:

The header’s hash

Return type:

int

property header_size

Each buffer file has a header. This property provides the size of the header in bytes (The normal value is 2000 bytes).

Returns:

Size in bytes.

Return type:

int

io_ports(db_idx: int, byte: int | None = None, bit: int | None = None)[source]

Deprecated method! This method provides the state of the io ports at the time the dateblock (indicated by its number db_idx) is written. It does not provide information wether the state of any io port changes within the datablock. This method will be replaced by a method which analyses the data within the sub data block, which is part of the data block header and provides information regarding the state of the io ports at a higher accuracy.

Parameters:

  • db_idx (int) – Index of the data block

  • byte (int, optional) – Number of io port socket [1, 2, 4]

  • bit (int, optional) – Number of bit [1, 2, 3, 4, 5, 6, 7, 8]

Raises:

  • ValueError – The bit argument requires the byte argument.

  • ValueError – The given byte has to be one out of [1, 2, 4].

  • ValueError – The given bit is out of range.

Returns:

byte with the appropriate bits set as an int

Return type:

int

io_ports_spec(spec: int, byte: int, bit: int)[source]

Deprecated method! This method provides the state of the io ports at the time the dateblock (indicated by the spec number within the data block) is written. It does not provide information wether the state of any io port changes within the datablock. This method will be replaced by a method which analyses the data within the sub data block, which is part of the data block header and provides information regarding the state of the io ports at a higher accuracy.

Parameters:

  • spec (int) – Index of the spectrum

  • byte (int) – Number of io port socket [1, 2, 4]

  • bit (int) – Number of bit [1, 2, 3, 4, 5, 6, 7, 8]

Raises:

  • ValueError – The bit argument requires the byte argument.

  • ValueError – The given byte has to be one out of [1, 2, 4].

  • ValueError – The given bit is out of range.

Returns:

byte with the appropriate bits set as an int

Return type:

int

property last_modification_time

Returns the Posix timestamp of last modification of the file in seconds since Thursday, January 1st 1970.

Returns:

The timestamp as a number in seconds.

Return type:

int

static log(data_arr, fft_log_shift, ad_bit_resolution)[source]

Method to logarithmize a data array. Usually when data are measured with the Optimizer the data are logarithmized. The problem with logarithms is that adding them up constitutes a multiplication of the non logarithmized data. So in order to carry out calculations such as calculating sums etc. it is crucial to use de-logarithmized data. Sometimes it is necessary to logarithmize data which were artificially created or measured without a logarithm.

Parameters:

  • data_arr (numpy ndarray of floats) – Array with non-logarithmized data.

  • fft_log_shift (int) – Base of the logarithm.

  • ad_bit_resolution (int) – The bit resolution (usually 16).

Returns:

Array with logarithmized data.

Return type:

numpy ndarray of floats

property max_amplitude_level

maximum possible amplitude of the buffer, relates to the buffer’s dtype :rtype: int, float

property metainfo

The buffer header properties as a dictionary.

Returns:

The metainfo dictionary

Return type:

dictionary

 1key = 'qassdata'
 2def keyword(key)
 3    return buffer.metainfo[key]
 4
 5# The above example leads to a runtime error if the keyword is not a
 6# key in the dictionary. It would be better to query this beforehand
 7# and to provide a default value, thus:
 8
 9def keyword(key):
10    if key in buffer.metainfo.keys:
11        return buffer.metainfo[key]
12    else:
13        return default_value
14
15# or shorter:
16
17def keyword(key):
18    return buffer.metainfo.get(key, default_value)
property partnumber: str

The string representation of the partnumber of the measurement. :rtype: str

property preamp_gain

The preamplifier setting in the multiplexer tab of the Analyzer4D software.

Return type:

int

property process

The process number.

Returns:

The process number.

Return type:

int

property process_date_time

Returns a timestamp of the creation of the file as a string.

Returns:

The timestamp in the form ‘yyyy-mm-dd hh:mm:ss’

Return type:

string

property process_measure_time_string

Returns the time in ISO format at measure start of the process. If epoctime (measure time) is not available due to an older buffer format, the creation time of the buffer is returned. Please note: The creation time may not be the same as epoc time. This is particularly true for compressed buffers which may be created much later!

Returns:

The timestamp in the form ‘yyyy-mm-dd hh:mm:ss’.

Return type:

string

property process_measure_timestamp

Returns the Posix timestamp at measure start of the process in milliseconds. If epoctime (measure time) is not available due to an older buffer format, the creation time of the buffer is returned. Please note: The creation time may not be the same as epoc time. This is particularly true for compressed buffers which may be created much later!

Returns:

The timestamp as a number in seconds.

Return type:

int

property process_time

Returns the Posix timestamp of creation of the file in seconds since Thursday, January 1st 1970. Please note that the creation time and measure time might be different.

Returns:

The timestamp as a number of seconds.

Return type:

int

property refEnergy

refEnergy provides a normalization factor for compressions. This makes sums of amplitudes of different compressions comparable to each other. See _signalNormalizationFactor for a detailed description.

Return type:

float

property ref_energy

alias for refEnergy()

property sample_count

The number of samples in the buffer. It is calculated by subtracting the header size and the data block size times the number of datablocks from the file size. This figure is divided by the bytes per sample to obtain the number of samples. The number is cast to an int as the division usually results in a float due to an incompletely filled final data block.

Returns:

Number of samples

Return type:

int

property spec_count

The number of spectra in the buffer. It is calculated by dividing the number of samples by the number of frequency bands.

Returns:

number of spectra

Return type:

int

property spec_duration

The property returns the time for one spectrum in nanoseconds.

Returns:

Time in nanoseconds.

Return type:

float

spec_to_time_ns(spec)[source]

The method calculates the time since measurement start for a given index of a spectrum.

Parameters:

spec (int) – The index of a spectrum

Returns:

time in nanoseconds

Return type:

float

property streamno: int | None

The index of a stream of an extra channel.

Extra channels in the analyzer software can map multiple different data streams to the same channel of the software. This makes them only distuinguishable by this property.

time_to_spec(time_ns)[source]

The method calculates the index of the nearest spectrum to the time given since the start of the measurement.

Parameters:

time_ns (float) – Time elapsed since measurement start in ns.

Returns:

index of the spectrum

Return type:

int

Filter buffers

qass.tools.analyzer.buffer_parser.filter_buffers(directory, filters)[source]

This function is currently part of the buffer parser module but is likely to be moved into a repository with tools. This function takes a directory with buffer files and a dictionary with fiter criteria and returns a list of buffer objects from this dictionary which fulfill the filter criteria.

Parameters:

directory – path to a directory with buffer files

:type string :param filters: dictionary with filters :type dictionary

Returns:

A list of buffer objects

Return type:

list

1from qass.tools.analyzer import buffer_parser as bp
2proc = range(1,100,1)
3# path contains the path to a directory containing buffer files
4buff = bp.filter_buffers(path, {'wanted_process': proc , 'datamode': bp.Buffer.DATAMODE.DATAMODE_FFT})

Warning

This is deprecated! It’s highly recommended to use the qass.tools.analyzer.buffer_metadata_cache.BufferMetadataCache instead.