-
Notifications
You must be signed in to change notification settings - Fork 5
BlackChirp Data Files
BlackChirp stores acquisition data in numbered "Experiments." Each experiment performed with the program is automatically assigned a number, which the program recalls between sessions and increments with each successive acquisition. Each experiment is saved in a folder according to its number, and the folder contains several different files which store different portions of the experimental data.
Experiment data can be located in the BC_DATADIR directory (for more information on changing this value, see the configuration instructions). By default, this is /home/data on Linux and Mac systems, and C:/data on Windows systems. Experiment z can be found in:
BC_DATADIR/blackchirp/experiments/x/y/z/
where x = floor(z/1000000) and y = floor(z/1000). In other words, x is the millions digit(s) of the experiment number, and y is the thousands digit(s). This scheme is used to prevent directories from being filled with large numbers of files, which degrades file system performance. Thus, each directory will contain no more than 1000 files or subdirectories. For instance, data files for experiment 12893 would be located at:
BC_DATADIR/blackchirp/experiments/0/12/12893/
Each experiment folder contains several different types of files, each of which records different information about an aspect of the experiment. While this may be slightly inconvenient from the point of view of the end user, breaking the experiment up in this manner allows for more economical data storage and more robust backward compatibility as features of the program involve. For convenience, a set of python scripts have been developed that are capable of reading in all of the data into an easy-to-use python object.
Brief descriptions of the data contained in each file type are given below.
The chirp file contains information about the upconversion chain and microwave chirp used with a CP-FTMW measurement in plain text format, so it can be examined with any text editor. Note that if an experiment did not involve CP-FTMW spectroscopy, this file will not be present. An example of the contents of a chirp file is shown below:
ChirpConfigAwgMult 1.0
ChirpConfigChirpInterval 20.000 μs
ChirpConfigMixerSideband -1.0
ChirpConfigNumChirps 1
ChirpConfigPostChirpProtection 0.300 μs
ChirpConfigPreChirpDelay 0.100 μs
ChirpConfigPreChirpProtection 0.010 μs
ChirpConfigTotalMult 4.0
ChirpConfigTxFreq 5760.000 MHz
ChirpConfigTxMult 2.0
Segment 4895.000 1520.000 0.5000
#Segment 26500.000 40000.000 0.5000
The file contains several key-value-unit sets delimited by tab characters; these refer to upconversion chain settings needed to convert desired chirp frequencies to raw AWG frequencies and vice-versa, as well as settings pertaining to the chirp's protection/enable pulses, number of chirps, and chirp interval. Following an empty line, a line starting with "Segment" contains the AWG starting frequency and ending frequency (both in MHz) and the duration of the segment in microseconds. If multiple segments were defined, then multiple such lines will appear. After another blank line, a corresponding "#Segment" line appears with the converted chirp frequencies (in MHz) and duration in microseconds.
This file can be loaded in the chirp configuration stage of the Experiment Creation wizard to ensure that a future acquisition uses the same parameters as a previous experiment. However, if the Rf configuration has been changed, then the chirp file will not be able to be loaded successfully, and the program will report an error.
The FID file contains information about all Free Induction Decays recorded during a CP-FTMW measurement in a custom binary format. This file cannot be meaningfully viewed with a text editor, but the python analysis scripts are capable of reading these files, as is BlackChirp itself. The .fid extension is used when the file contains only a single averaged oscilloscope record (which may contain multiple time frames), and the .mfd (multiple FID) extension is used when there are multiple averaged oscilloscope records, such as in a segmented CP-FTMW scan that steps through multiple LO frequencies.
TODO: Add link to developer guide with more details about file format
The header file contains information about the experiment as a whole in plain text format (i.e., it can be viewed by any text editor). Each entry is in the form of a key-value-unit set delimited by tab characters and terminated with a newline. An example of a small portion of the output is below:
AutosaveInterval 20000 shots
AuxDataInterval 300 s
ChirpConfigAwgMult 1.0
ChirpConfigChirpInterval 19.000 μs
ChirpConfigMixerSideband -1.0
ChirpConfigNumChirps 2
ChirpConfigPostChirpProtection 0.250 μs
ChirpConfigPreChirpDelay 0.150 μs
ChirpConfigPreChirpProtection 0.050 μs
ChirpConfigTotalMult 4.0
ChirpConfigTxFreq 5760.000 MHz
ChirpConfigTxMult 2.0
FlowConfigPressureAverage 0.000 kTorr
FlowConfigPressureControlMode false
FlowConfigPressureSetpoint 0.000 kTorr
FtmwConfigCompletedShots 18
FtmwConfigEnabled true
FtmwConfigFidVMult 9.1552734375e-06 V
FtmwConfigLoFrequency 41000.000000 MHz
FtmwConfigPhaseCorrection true
FtmwConfigSideband 0
FtmwConfigTargetTime 2016-05-09T18:27:46
FtmwConfigType 1
FtmwScopeByteOrder LittleEndian
FtmwScopeBytesPerPoint 2
FtmwScopeFastFrame true
FtmwScopeFidChannel 2
FtmwScopeNumFrames 2
FtmwScopeRecordLength 500000
FtmwScopeSampleRate 50.000 GS/s
FtmwScopeSummaryFrame true
FtmwScopeTriggerChannel 4
FtmwScopeTriggerDelay 0 s
FtmwScopeTriggerSlope FallingEdge
FtmwScopeVerticalOffset 0.000 V
FtmwScopeVerticalScale 0.060 V/div
IOBoardConfigAnalog.0.Enabled false
IOBoardConfigAnalog.0.Name TestAnalog0Plot
IOBoardConfigAnalog.0.Plot false
IOBoardConfigAnalog.1.Enabled false
IOBoardConfigAnalog.1.Name ain1
IOBoardConfigAnalog.1.Plot false
TODO
The log file contains all messages emitted by the program to the log tab. It is in a plain text format that can be read with any text editor. BlackChirp inserts [HIGHLIGHT], [WARNING], [ERROR], or [DEBUG] at the beginning of the message as appropriate to indicate what text decorations to use when rendering the text in the log. The log will additionally record information about how autosave snapshots were handled and where the experiment was exported in ASCII format. An example is given below:
Wed Sep 2 17:16:37 2015: [HIGHLIGHT] Experiment 5 started.
Wed Sep 2 17:20:26 2015: [HIGHLIGHT] Experiment 5 complete.
Wed Sep 2 17:20:31 2015: Finalizing snapshots....
Wed Sep 2 17:20:31 2015: All snapshots kept.
Wed Sep 2 17:20:31 2015: Remainder of shots kept.
Wed Sep 2 17:20:31 2015: Final number of shots: 768
Autosave snapshots are encoded in two files: a plaintext .snp file that contains information about how many and what types of snapshots have been recorded, and a -snapX.fid~ file that contains the actual snapshot data in binary format. These files should never be modified; they are used internally by BlackChirp to enable its snapshot functionality. After the snapshots are "resolved" in the BlackChirp program, these files will be removed.
TODO: Add link to documentation on snapshot functionality.
The time data file contains all auxiliary data that were logged during an experiment, written in a plain text format. This includes both data that were plotted on the "Tracking" tab as well as other data that may not have been plotted (e.g., timestamps or digital input signals). An example is shown below for experiment 11:
#Alias Test2 flow.1
#Alias Test3 flow.2
#PlotData
Test2_11 Test3_11 ftmwShots_11 pressure_11
1.036000e+01 4.900000e+00 0.000000e+00 2.000000e+00
9.900000e+00 5.215000e+00 2.300000e+01 2.000000e+00
1.005000e+01 5.210000e+00 4.800000e+01 2.000000e+00
1.026000e+01 4.980000e+00 7.300000e+01 2.000000e+00
1.013000e+01 5.005000e+00 9.800000e+01 2.000000e+00
1.025000e+01 4.880000e+00 1.230000e+02 2.000000e+00
9.810000e+00 5.075000e+00 1.480000e+02 2.000000e+00
9.910000e+00 4.825000e+00 1.730000e+02 2.000000e+00
1.041000e+01 5.100000e+00 1.980000e+02 2.000000e+00
1.022000e+01 4.950000e+00 2.230000e+02 2.000000e+00
9.710000e+00 4.955000e+00 2.480000e+02 2.000000e+00
9.810000e+00 4.770000e+00 2.730000e+02 2.000000e+00
9.550000e+00 5.035000e+00 2.980000e+02 2.000000e+00
9.600000e+00 5.135000e+00 3.230000e+02 2.000000e+00
1.035000e+01 5.065000e+00 3.480000e+02 2.000000e+00
9.660000e+00 4.750000e+00 3.730000e+02 2.000000e+00
1.006000e+01 4.915000e+00 3.980000e+02 2.000000e+00
1.020000e+01 4.765000e+00 4.230000e+02 2.000000e+00
1.037000e+01 4.830000e+00 4.480000e+02 2.000000e+00
1.018000e+01 5.065000e+00 4.730000e+02 2.000000e+00
9.750000e+00 4.980000e+00 4.980000e+02 2.000000e+00
#NoPlotData
exptTimeStamp_11
2016-04-26T10:12:10
2016-04-26T10:12:15
2016-04-26T10:12:20
2016-04-26T10:12:25
2016-04-26T10:12:30
2016-04-26T10:12:35
2016-04-26T10:12:40
2016-04-26T10:12:45
2016-04-26T10:12:50
2016-04-26T10:12:55
2016-04-26T10:13:00
2016-04-26T10:13:05
2016-04-26T10:13:10
2016-04-26T10:13:15
2016-04-26T10:13:20
2016-04-26T10:13:25
2016-04-26T10:13:30
2016-04-26T10:13:35
2016-04-26T10:13:40
2016-04-26T10:13:45
2016-04-26T10:13:50
The file may contain several #Alias lines that provide a mapping between user-defined "pretty" labels (e.g., gas channel names) and BlackChirp keys (e.g., flow.1). After empty lines, the line #PlotData signifies the beginning of data entries that were plotted on the "Tracking" tab. The data themselves are tab-delimited plain text, starting with a column header with _X appended, where X is the experiment number. After the #PlotData section comes a #NoPlotData section in which data that were not plotted are stored.
Choosing the "Export Experiment" action from the user interface generates a plain text version of all of the experiment data (currently, LIF data are not exported). The size of the ASCII file is usually much larger than the size of the data contained in the experiment folder, sometimes by as much as a factor of 10, so use caution when exporting many experiments.
The exported text file is essentially a concatenation of the header file, the log file, the fid file, and the time data file. Each file's contents is separated by two newline characters, and each line of the header and log file contents is preceded with a '#' character for easier processing. The FID data are converted to a text representation and printed in a column (or in tab-delimited columns if more than one FID was recorded).
Kyle N. Crabtree, [email protected]