autorino’s exemple cookbook
Metadata disclaimer
autorino assumes that the metadata in your RINEX’s header are always wrong.
Thus, it will always update the metadata based on an external source, such as sitelogs or receiver/antenna details given in the site configuration file.
NB: after the conversion step, for receiver’s serial number and firmware version, a comparison is made between the exteral metadata source and the newly converted RINEX’s header.
If a difference is detected, a warning is raised, because the RINEX values must be the correct ones.
Nevertheless, the RINEX header will be updated with the external metadata (the warning is simply an encouragement to change you external metadata source).
Receipe #1: run an autorino workflow
Call autorino_cfgfile_run to run an autorino workflow i.e. a set of steps.
The workflow is defined in a YAML site configuration file.
See the next chapters configuration files in a nutshell &
configuration files details for more details.
Synopsis
$ autorino_cfgfile_run --help
250430T23:55:24.696|[32mI[0m|[32mread_env [0m|Loading environment configfile: /home/psakicki/CODES/IPGP/autorino_configfiles/configfiles/Paris/env/autorino_env_PS_TPX1_1a.yml[0m
usage: autorino_cfgfile_run [-h] -c CONFIG
[-i INCLUDE_CONFIG [INCLUDE_CONFIG ...]]
[-s START] [-e END] [-p PERIOD]
[-l LIST_SITES [LIST_SITES ...]] [-is]
[-sc STEPS_SELECT_LIST [STEPS_SELECT_LIST ...]]
[-es] [-f]
Assisted Unloading, Treatment and Organization of RINEX observations
options:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
The input site configuration file or directory of
sites configuration files. If a directory is provided,
all files ending with '.yml' will be used.
-i INCLUDE_CONFIG [INCLUDE_CONFIG ...], --include_config INCLUDE_CONFIG [INCLUDE_CONFIG ...]
The include configuration files to be used for
development or advanced purposes. If a list is
provided, all files in the list will be included.
These files override the `include` section of the main
configuration file.
-s START, --start START
The start date for the epoch range. Can be a date e.g.
'2025-01-01', '2025-001' or a literal e.g. '2 days
ago'. Can also be a list; if so, each epoch is
considered separately. Can also be a file path; if so,
the file contains a list of start epochs. Default is
None.
-e END, --end END The end date for the epoch range. Can be a date e.g.
'2025-01-01', '2025-001' or a literal e.g. '2 days
ago'. Default is None.
-p PERIOD, --period PERIOD
The period for the epoch range i.e. the sampling of
the files: daily = '1D', hourly = '1H', 15 minutes =
'15M'. Default is '1D'.
-l LIST_SITES [LIST_SITES ...], --list_sites LIST_SITES [LIST_SITES ...]
list of site identifiers ('site_id') in the config
fileto filter the configuration files. If provided,
only configurations for sites in this list will be
processed. Default is None.
-is, --ignore_sites If True, the sites in --list_sites will be ignored.It
is the opposed behavior of the regular one using
list_sites.Default is False.
-sc STEPS_SELECT_LIST [STEPS_SELECT_LIST ...], --steps_select_list STEPS_SELECT_LIST [STEPS_SELECT_LIST ...]
A list of selected steps to be executed. If not
provided, all steps in 'steps_list' will be executed.
Accepted steps are: 'download', 'convert', 'splice',
'split'. Default is None.
-es, --exclude_steps_select
If True the selected steps indicated in
step_select_list are excluded. It is the opposite
behavior of the regular one using steps_select_list.
Default is False.
-f, --force If True, the steps will be executed even if the output
files already exist. Overrides the 'force' parameters
in the configuration file. Default is False.
Examples:
* run all the config files within cfgfiles_dir directory, using a main config file and per default epoch ranges:
autorino_cfgfile_run -c cfgfiles_dir -m main_cfg.yml
* run the config file site_cfg.yml from the 1st January 2025 for a range of 10 days:
autorino_cfgfile_run -c site_cfg.yml -s 2025-01-01 -e '10 days ago'
* run download and convert steps only for HOUZ00GLP & BORG00REU sites only:
autorino_cfgfile_run -c cfgfiles_dir -m main_cfg.yml -l HOUZ00GLP BORG00REU -sc download convert
Examples
We set the configuration file path in an environment variable $ARO_CFGFILE.
ARO_CFGFILE=${AUTORINO_DIR}/configfiles/site/autorino_site_cfg.yml
Launching for specific dates
Run all config files in the sites/terrestrial folder for a 10-day period in the past starting from January 1, 2025
autorino_cfgfile_run -s '2025-01-01' -e '10 day ago' -c ${ARO_CFGFILE}
Launching specific steps
Use -ss/--select_steps. For example, to run only the download and conversion steps:
autorino_cfgfile_run -ss download convert -c ${ARO_CFGFILE}
Excluding specific steps
Use -ss/--select_steps to select the steps to exclude, then activate -es/--exclude_steps_select.
autorino_cfgfile_run -ss download -es -c ${ARO_CFGFILE}
Launching for specific sites
Use -l/--list_sites to run a specific site. For example, to run the site SOUF00GLP & BULG00GLP:
autorino_cfgfile_run -l SOUF00GLP BULG00GLP -c ${ARO_CFGFILE}
Receipe #2: convert RAW to RINEX files
Call autorino_convert_rnx_ to convert RAW files to RINEX files.
Synopsis
$ autorino_convert_rnx --help
250501T00:22:43.465|[32mI[0m|[32mread_env [0m|Loading environment configfile: /home/psakicki/CODES/IPGP/autorino_configfiles/configfiles/Paris/env/autorino_env_PS_TPX1_1a.yml[0m
usage: autorino_convert_rnx [-h] -i INP_RAWS [INP_RAWS ...] -o OUT_DIR
[-t OUT_STRUCTURE] [-m METADATA] [-frnx] [-fraw]
[-l] [-tmp TMP_DIR] [-log LOG_DIR]
[-rimo RINEXMOD_OPTIONS] [-ro RAW_OUT_DIR]
[-rt RAW_OUT_STRUCTURE]
Convert RAW files to RINEX.
options:
-h, --help show this help message and exit
-i INP_RAWS [INP_RAWS ...], --inp_raws INP_RAWS [INP_RAWS ...]
The input RAW files to be converted Possible inputs
are: * one single RAW file path * a comma-separated
(,) list of RAW paths * a text file containing a list
of RAW paths (then --list_file_input must be
activated) * a directory containing RAW files
-o OUT_DIR, --out_dir OUT_DIR
The output directory where the converted files will be
stored
-t OUT_STRUCTURE, --out_structure OUT_STRUCTURE
The structure of the output directory. If provided,
the converted files will be stored in a subdirectory
of out_dir following this structure. See README.md for
more information. Typical values are '<SITE_ID4>/%Y/'
or '%Y/%j/.
-m METADATA, --metadata METADATA
The metadata to be included in the converted RINEX
files. Possible inputs are: * list of string (sitelog
file paths) * single string (single sitelog file path)
* single string (directory containing the sitelogs)
-frnx, --force_rnx Force the conversion even if the output files already
exist
-fraw, --force_raw Force the RAW file archiving even if the output files
already exist
-l, --list_file_input
If set to True, the input RAW files are provided as a
list in a text file
-tmp TMP_DIR, --tmp_dir TMP_DIR
The temporary directory used during the conversion
process. If not provided, it defaults to
<$HOME>/tmp_convert_rnx.
-log LOG_DIR, --log_dir LOG_DIR
The directory where logs will be stored. If not
provided, it defaults to tmp_dir
-rimo RINEXMOD_OPTIONS, --rinexmod_options RINEXMOD_OPTIONS
The options for modifying the RINEX files during the
conversion. The options must be provided in a
dictionnary represented as a string e.g. '{longname:
False, filename_style: basic}' Defaults to None
-ro RAW_OUT_DIR, --raw_out_dir RAW_OUT_DIR
Directory where RAW files will be archived. No
deletion will occur, your RAW files are sacred.
-rt RAW_OUT_STRUCTURE, --raw_out_structure RAW_OUT_STRUCTURE
Structure for archiving RAW files. Defaults to the
same structure as the output directory if not
provided.
Receipe #3: Check a configuration file
Call autorino_cfgfile_check to check the import of a configuration file, especially its include.
Synopsis
$ autorino_cfgfile_check --help
250501T00:28:05.803|[32mI[0m|[32mread_env [0m|Loading environment configfile: /home/psakicki/CODES/IPGP/autorino_configfiles/configfiles/Paris/env/autorino_env_PS_TPX1_1a.yml[0m
usage: autorino_cfgfile_check [-h] -c CONFIG
[-i INCLUDE_CONFIG [INCLUDE_CONFIG ...]]
[-o OUTPUT]
Check autorino configuration file
options:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
The input site configuration file or directory of
sites configuration files. If a directory is provided,
all files ending with '.yml' will be used.
-i INCLUDE_CONFIG [INCLUDE_CONFIG ...], --include_config INCLUDE_CONFIG [INCLUDE_CONFIG ...]
The main configuration file to be used.
-o OUTPUT, --output OUTPUT
The output path where to write the checked
configuration.
Receipe #4: Convert RAW file to RINEX in API mode
The most basic and common operation that autorino can perform is to convert some RAW files to RINEX.
convert_rnx function minimal example
import autorino.common as arocmn
import glob
### Find all BINEX files in a folder
p = "/home/user/where_your/raw_data/are_stored/"
l = glob.glob(p,"*BNX")
### Define the output folder
out_dir = "/home/user/where_your/rinex_data/will_be_saved/"
tmp_dir = out_dir
### Call the conversion function
arocmn.convert_rnx(l,out_dir,tmp_dir)
convert_rnx function docstring
- autorino.api.convert_rnx(inp_raws, out_dir, out_structure='%Y/%j', tmp_dir=None, log_dir=None, rinexmod_options=None, metadata=None, force_rnx=False, force_raw=False, raw_out_dir=None, raw_out_structure=None, processes=1, filter_prev_tables=False)[source]
Frontend function that performs RAW > RINEX conversion.
- Parameters:
inp_raws (list) – The input RAW files to be converted. The input can be: * a python list * a text file path containing a list of files * a tuple containing several text files path * a directory path.
out_dir (str) – The output directory where the converted files will be stored.
out_structure (str, optional) – The structure of the output directory. If provided, the converted files will be stored in a subdirectory of out_dir following this structure. See README.md for more information. Typical values are ‘<SITE_ID4>/%Y/’ or ‘%Y/%j/’. Default value is ‘%Y/%j/’.
tmp_dir (str, optional) – The temporary directory used during the conversion process. If not provided, it defaults to <out_dir>/tmp_convert_rnx. Defaults to None.
log_dir (str, optional) –
- The directory where logs will be stored. If not provided, it defaults to tmp_dir.
Defaults to None.
rinexmod_options (dict, optional) – The options for modifying the RINEX files during the conversion. Defaults to None.
metadata (str or list, optional) –
The metadata to be included in the converted RINEX files. Possible inputs are:
list of string (sitelog file paths),
single string (single sitelog file path)
single string (directory containing the sitelogs)
list of MetaData objects
single MetaData object.
Defaults to None.
force_rnx (bool, optional) – If set to True, the conversion will be forced even if the output files already exist. Defaults to False.
force_raw (bool, optional) – If set to True, the RAW file archiving will be forced even if the output files already exist. Defaults to False.
raw_out_dir (str, optional) – Directory where RAW files will be archived. No move/delete will occur, your input RAW files are sacred. Defaults to None.
raw_out_structure (str, optional) – Structure for archiving RAW files. Defaults to out_structure if not provided.
processes (int, optional) – Number of processes to use for parallel conversion. Default is 1.
filter_prev_tables (bool, optional) – If True, filters and skip previously converted files with tables stored in the tmp tables directory. Default is False.
- Return type:
None