geodezyx.operational package

Subpackages

Submodules

geodezyx.operational.anubis_frontend module

Created on Tue Jul 5 15:48:26 2022

@author: psakic

geodezyx.operational.anubis_frontend.anubis_runner(rnx_inp, out_dir_main, xml_config_generic, period=None, interval=None, dry_run=False, download_nav=True, download_sp3=False, force=False, anubis_path='/opt/gnss_softs/bin/anubis')

Run an Anubis quality check. Designed for Anubis 3.3. Works with Anubis 3.7 at least.

Parameters:

  • rnx_inp (str or list) –

    Input RINEXs. can be a RINEX path list, or the path of the parent archive directory

    Note that this function is optimized for the batch processing of several RINEXs. If you want to process a single RINEX, gives its path in a list ([rnx_inp])

  • out_dir_main (str) – output main directory.

  • xml_config_generic (str) – path of the generic XML configuration file. will be stored in <out_dir_main>/inp See note bellow to find some exemples

  • period (None or int) – nominal file period in the RINEX (in sec) if None is given, guess based on the RINEX name The default is None.

  • interval (None or int) – nominal data interval in the RINEX (in sec) if None is given, guess based on the RINEX name The default is None.

  • dry_run (bool, optional) – if True, do not run the Anubis QC. Anubis QC results are stored in <out_dir_main>/out The default is False.

  • download_nav (bool, optional) – Download automatically the Broadcast navigation files. will be stored in <out_dir_main>/nav The default is True.

  • download_sp3 (bool, optional) – Download automatically the SP3 orbit files. CODE’s MGEX or REPRO3 before 2018 are used per default. will be stored in <out_dir_main>/nav The default is False.

  • force (bool, optional) – Per default, skip anubis execution if a xtr file already exists in out_dir_main The default is False.

  • anubis_path (str, optional) – path of the Anubis executable. The default is “/opt/gnss_softs/bin/anubis”

Returns:

xml_cfg_list – list of the generated XML config files.

Return type:

list

Note

Generic Configuration files for Anubis version 2 and 3 can be found here:

<...>/geodezyx/exemples/anubis_configfiles

or directly on the GeodeZYX’s toolbox GitHub repository:

https://github.com/GeodeZYX/geodezyx-toolbox/tree/master/geodezyx/000_exemples/anubis_configfiles

geodezyx.operational.cluster_gfz module

@author: psakic

This sub-module of geodezyx.operational contains functions to send runs to the GFZ’s cluster in a batch mode.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.cluster_gfz.cluster_GFZ_run(commands_list, bunch_on_off=True, bunch_job_nbr=10, bunch_wait_time=600, bj_check_on_off=True, bj_check_mini_nbr=2, bj_check_wait_time=120, bj_check_user='auto', add_cjob_cmd_prefix=True, wait_sleeping_before_launch=5)
Parameters:

  • commands_list (list of str) – List of commands you want to run (one command per job).

  • bunch_on_off (bool, optional) – If False, send all the jobs to the cluster at once. If True, just send <bunch_job_nbr> of them. See the options below. The default is True.

  • bunch_job_nbr (int, optional) – number of jobs which will be sent to the cluser at once. The default is 10.

  • bunch_wait_time (int, optional) – Minimal time between two bunch runs in sec. The default is 600.

  • bj_check_on_off (bool, optional) – Do a check before a new bunch run, if previous jobs are still running. The default is True.

  • bj_check_mini_nbr (int, optional) – The fuctions will wait <bj_check_wait_time> sec more if <bj_check_mini_nbr> or more jobs are running. The default is 2.

  • bj_check_wait_time (int, optional) – wait time between two checks in sec. The default is 120.

  • bj_check_user (str, optional) – username you want to check in the batchjobs. The default is “auto” i.e. your own username.

  • add_cjob_cmd_prefix (bool, optional) – If, the input commands in command_list do not contain the cjob prefix command, it will be added automatically The default is True.

  • wait_sleeping_before_launch (int, optional) – Waiting time between two successive runs. (To cancel the run if necessary) The default is 5.

Return type:

None.

geodezyx.operational.cluster_gfz.number_job_user(bj_check_user=None, verbose=True)

Internal function for sleep_job_user

geodezyx.operational.cluster_gfz.sleep_job_user(bj_check_user=None, minjob=20, bj_check_wait_time=20)

Internal function for cluster_GFZ_run

geodezyx.operational.download_cddis module

@author: mansur This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_cddis.download_rp3_cddis(week_ini=None, ac=None, fmt=None, out_path=None)

Download sp3 weekly files for repro3 products, from cddis (NASA). Note: For the moment is only for sp3 and clk.

Parameters:

  • week_ini (GPS week for download)

  • fmt (searching format file e.g. sp3)

  • ac (name of the anlisys center)

  • out_path (path where the products will be download)

Return type:

No returns.

geodezyx.operational.download_cddis.list_daily_files_ftp(ftp_dir_list=None, fmt='sp3', ac='COD', week=1900, dayw=0)

List files inside one directory in a ftp serve. Note that this uses an anonymous user.

Parameters:

  • ftp_dir_list (list with the files in the ftp (comming from the previous fct: list_file_in_ftp))

  • fmt (searching format file e.g. sp3)

  • ac (name of the anlisys center)

  • week (gps week)

  • day (day of the week)

Return type:

Return a list with the the information about the directory.

geodezyx.operational.download_cddis.list_file_in_ftp(email='mansur@gfz-potsdam.de', host='gdc.cddis.eosdis.nasa.gov', directory='gnss/products/repro3/1900')

List files inside one directory in a ftp serve. Note that this uses an anonymous user.

Parameters:

  • email (login email)

  • host (ftp server)

  • directory (directory inside the server)

Return type:

Return a list with the the information about the directory.

geodezyx.operational.download_dropbox module

Created on Thu Feb 2 14:37:57 2023

@author: psakic

Based on https://practicaldatascience.co.uk/data-science/how-to-use-the-dropbox-api-with-python

The Dropbox app parameters are here https://www.dropbox.com/developers

The important permission parameters are:

sharing.write the other can be read only

reset the token after changing the permissions!

geodezyx.operational.download_dropbox.dropbox_connect(dropbox_access_token)

Create a connection to Dropbox.

geodezyx.operational.download_dropbox.dropbox_list_files(dbx, path)
Parameters:

  • dbx (dropbox.dropbox_client.Dropbox) – The dropbox_connect output object.

  • path (str) – the path of your Dropbox folder. (the root folder with your name not included) e.g. dropbox_path = “/split_2021_c/” dropbox_path = “/TE/RINEX 3.04/” + str(year) dropbox_path = “/RINEX 3.04/” + str(year)

Returns:

  • df (DataFrame) – a Pandas dataframe of files in a given Dropbox folder path in the Apps directory.

  • files (list of dropbox.files.FileMetadata) – list of dropbox.files.FileMetadata.

geodezyx.operational.download_dropbox.dropbox_wget_cmds(DFfiles, dbx, out_dir_dwnld_files, out_dir_cmd_list, cmd_list_suffix, run_get_temp_link=False, run_get_perma_link=True, teria_archive=True, start_date=datetime.datetime(1980, 1, 1, 0, 0), end_date=datetime.datetime(2099, 1, 1, 0, 0))

Generate URL and wget commands to download files from a dropbox Especially to manage Teria’s RINEXs but no only.

Parameters:

  • DFfiles (DataFrame) – Files DataFrame outputed from dropbox_list_files.

  • dbx (dropbox.dropbox_client.Dropbox) – The dropbox_connect output object.

  • out_dir_dwnld_files (str) – directory for the future downloaded files.

  • out_dir_cmd_list (str) – directory for the URL list/wget commands.

  • cmd_list_suffix (str) – suffix for the URL list/wget commands files.

  • run_get_temp_link (bool, optional) – request temporary link. The default is False.

  • run_get_perma_link (bool, optional) – request permanent link. Recommeded. Overrides run_get_temp_link. The default is True.

  • teria_archive (bool, optional) – manage time for the Teria archive. The default is True.

  • start_date (datetime, optional) – start date. The default is dt.datetime(1980,1,1).

  • end_date (datetime, optional) – end date. The default is dt.datetime(2099,1,1).

Returns:

  • WgetList (list) – list of wget commands.

  • UrlDF (DataFrame) – DataFrame of the download URLs.

geodezyx.operational.download_find_files module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_find_files.find_igs_products_files(parent_dir, file_type, ACs, date_start, date_end=None, recursive_search=True, severe=True, compressed='incl', regex_old_naming=True, regex_new_naming=True, regex_igs_tfcc_naming=True, add_weekly_file=False, add_hourly_file=False)

Find all product files in a parent folder which correspond to file type(s), AC(s) and date(s)

Parameters:

parent_dir (str or list of str) – The parent directory (i.e. the archive) where files are stored can be a string (path of the archive) or a list of file paths (given by the function utils.find_recursive) in order to gain time

file_typestr or list of str

File type(s) researched (sp3, erp, clk …) can be a list of string for several file type paths or a string like ‘sp3’ if only one file type is researched

ACs‘all’ or str or list of str

AC(s) researched can be a list of string for several ACs or a string like ‘gfz’ if only one AC is researched if ‘all’, search for all the ACs

date_startdt.datetime or 2-tuple list of int

begining of the time period researched can be a datetime or a 2-tuple (wwww,d) e.g. (1990,0)

date_endNone or dt.datetime or 2-tuple list of int

end of the time period researched can be a datetime or a 2-tuple (wwww,d) e.g. (1990,0) if None, then only date_start is researched

severebool

If True, raises an exception if something goes wrong

compressedstr

How the compressed files are handled “incl”: include the compressed files “only”: only consider the compressed files “excl”: exclude the compressed files

regex_old_namingbool

Handle old naming format

regex_new_namingbool

Handle new naming format

regex_igs_tfcc_namingbool

Handle TFCC specific format (for SINEX files)

add_weekly_filebool

Also handle the weekly file (day 7) Implemented only for the old naming format (for the moment)

add_hourly_filebool

Also handle ultra rapide hourly file Implemented only for the new naming format

Returns:

Files_select_cumul_list – List of files found

Return type:

list

geodezyx.operational.download_find_files.read_rinex_list_table(rnx_list_inp)

Generate a Table from a RINEX list

Parameters:

rnx_list_inp (str, list or iterable) – RINEX list. Can be a Python iterable (a list) or a path to a list file (a string)

Returns:

df – a RINEX Table.

Return type:

DataFrame

Note

From script …/geodezyx_toolbox_PS_perso_scripts/IPGP_OVS/rinex_lister/rinex_list_2019_2022_mk01.py

geodezyx.operational.download_find_files.rinex_finder(main_dir, short_name=True, long_name=True, gfz_godc_name=True, compressed=None, specific_sites=[], start_epoch=None, end_epoch=None)

Find RINEX files in a specified directory and filter them based on various criteria.

Parameters:

  • main_dir (str) – Main directory where the RINEX files are stored. The directory can contain a wildcard ‘*’, ‘?’, etc… The directory can contain date alias ‘%’, like ‘%Y’, ‘%j’, etc. If the main_dir contains a an date alias and both start_epoch and end_epoch are defined, You can indicate a more precise directory. (e.g. main_dir = “/path/to/data/*/%Y/%j/”) NB: The day level is the maximum resolution for the wildcard.

  • short_name (bool, optional) – Check if the pattern matches a short name RINEX. The default is True.

  • long_name (bool, optional) – Check if the pattern matches a long name RINEX. The default is True.

  • gfz_godc_name (bool, optional) – Check if the pattern matches a GFZ’s GODC (GNSS Operational Data Center) internal long name RINEX. The default is True.

  • compressed (bool or None, optional) – Check if the pattern matches a compressed RINEX (True) or not (False). If None, does not matter (return both compressed or not).

  • specific_sites (list, optional) – Filter only those specific sites. The default is [].

  • start_epoch (datetime, optional) – Start date for filtering the RINEX files.

  • end_epoch (datetime, optional) – End date for filtering the RINEX files.

Returns:

files_rnx_lis – List of found RINEX files.

Return type:

list

Notes

This function is very similar to geodetik.rinex_lister, gins_runner.get_rinex_list, and operational.multi_finder_rinex. However, this one is the most recent and elaborated (July 2022) and should be used in priority.

geodezyx.operational.download_prods module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_prods.download_gnss_products(archive_dir, startdate, enddate, AC_names=('wum', 'cod'), prod_types=('sp3', 'clk'), remove_patterns=('ULA',), archtype='week', new_name_conv=True, parallel_download=4, archive_center='ign', mgex=False, repro=0, sorted_mode=False, return_also_uncompressed_files=True, ftp_download=False, dow_manu=False)

Download GNSS products from different IGS data centers

Parameters:

  • archive_dir (str) – the parent directory where the products will be stored.

  • startdate (datetime) – the start date in regular calendar date.

  • enddate (datetime) – the end date in regular calendar date..

  • AC_names (tuple, optional) – the names of the wished analysis centers. It also control the product’s lattency with the new naming convention: simply add it completly in the AC name e.g. IGS0OPSRAP The default is (“wum”,”cod”).

  • prod_types (tuple, optional) – the wished products. The default is (“sp3”,”clk”).

  • remove_patterns (tuple, optional) – the patterns you want to exclude. The default is (“ULA”,).

  • archtype (str, optional) – structure of the local archive sub-directories. see effective_save_dir_orbit function for more details. The default is ‘week’. an alternatiove can be ‘year/doy’.

  • new_name_conv (bool, optional) – Also handle the new name convention. The default is True.

  • parallel_download (int, optional) – control parallel download. The default is 4.

  • archive_center (TYPE, optional) – name of the IGS’s archive/data center. The default is ‘ign’.

  • mgex (bool, optional) – get MGEX products. The default is False.

  • repro (int, optional) – get repro products. The default is 0 i.e. operational products.

  • sorted_mode (bool, optional) – sort the download or not. The default is False.

  • return_also_uncompressed_files (bool, optional) – in the final list output, return also already downloaded and uncompressedfiles. The default is True.

  • ftp_download (bool, optional) – DESCRIPTION. The default is False.

  • dow_manu (int or bool or None, optional) – Control the download for weekly files dow_manu = False, no dow manu, consider the converted dow from the time span, regular case dow_manu = None, no dow in the REGEX, the crawler will search only for the week dow_manu = 0 or 7, the dow in question The default is False.

Returns:

list of the local files’s paths.

Return type:

list

Note

The new naming convention has been fully adopted since GPS Week 2238-0

to control the lattency with the new naming convention, simply add it completly in the AC name e.g. IGS0OPSRAP

geodezyx.operational.download_prods.multi_downloader_orbs_clks(archive_dir, startdate, enddate, calc_center='igs', sp3clk='sp3', archtype='year/doy', parallel_download=4, archive_center='ign', repro=0, sorted_mode=False, force_weekly_file=False, return_also_uncompressed_files=True)

Download IGS products. Can manage MGEX products too (see archive_center argument)

Parameters:

  • archive_dir (str) – Parent archive directory where files will be stored

  • enddate (startdate &) – Start and End of the wished period

  • calc_center (str or list of str) – calc_center can be a string or a list, describing the calc center e.g. ‘igs’,’grg’,’cod’,’jpl’ …

  • sp3clk (str) –

    Product type, can handle :

    ’clk’

    ’clk_30s’

    ’sp3’

    ’snx’

    ’sum’

    ’erp’

    ’bia’

  • archive_center (str) –

    server of download, “regular” IGS or MGEX, can handle :

    ’cddis’

    ’cddis_mgex’

    ’cddis_mgex_longname’

    ’ign’

    ’ign_mgex’

    ’ign_mgex_longname’

    ’gfz_local’

  • archtype (str) –

    string describing how the archive directory is structured, e.g :

    stat

    stat/year

    stat/year/doy

    year/doy

    year/stat

    week/dow/stat

    … etc …

  • repro (int) – number of the IGS reprocessing (0 = routine processing)

  • sorted_mode (bool) –

    if False:

    using the map multiprocess fct so the download order will be scrambled

    if True:

    using the apply multiprocess fct so the download order will be in the chrono. order

    The scrambled (False) is better, bc. it doesn’t create zombies processes

Returns:

localfiles_lis – list of downloaded products paths

Return type:

list of str

geodezyx.operational.download_prods.multi_downloader_orbs_clks_2(**kwargs)
geodezyx.operational.download_prods.orbclk_long2short_name(longname_filepath_in, rm_longname_file=False, center_id_last_letter=None, center_manual_short_name=None, force=False, dryrun=False, output_dirname=None)

Rename a long naming new convention IGS product file to the short old convention Naming will be done automaticcaly based on the 3 first charaters of the long AC id e.g. CODE => cod, GRGS => grg, NOAA => noa …

Parameters:

  • longname_filepath_in (str) – Full path of the long name product file

  • rm_longname_file (bool) – Remove the original long name product file

  • center_id_last_letter (str) – replace the last letter of the short AC id by another letter (see note below)

  • center_manual_short_name (str) – replace completely the long name with this one overrides center_id_last_letter

  • force (bool) – if False, skip if the file already exsists

  • dryrun (bool) – if True, don’t rename effectively, just output the new name

  • output_dirname (str) – directory where the output shortname will be created if None, will be created in the same folder as the input longname

Returns:

shortname_filepath – Path of the short old-named product file

Return type:

str

Note

if you rename MGEX orbits, we advise to set center_id_last_letter=”m” the AC code name will be changed to keep a MGEX convention (but any other caracter can be used too)

e.g. for Bern’s products, the long id is CODE

if center_id_last_letter=None, it will become cod, if center_id_last_letter=m, it will become com

geodezyx.operational.download_rinex module

Created on 23/04/2024 21:31:45

@author: psakic

geodezyx.operational.download_rinex.crawl_ftp_files(table, sftp='auto', user=None, passwd=None, path_ftp_crawled_files_save=None, path_all_ftp_files_save=None, force=False)

filter the table with download_gnss_rinex with an optimized FTP crawl

geodezyx.operational.download_rinex.download_gnss_rinex(statdico, output_dir, startdate, enddate, archtype='stat', parallel_download=4, user='anonymous', passwd='anonymous@isp.com', path_ftp_crawled_files_save=None, path_ftp_crawled_files_load=None, skip_crawl=False, path_all_ftp_files_save=None, quiet_mode=False, final_archive_for_sup_check=None, force=False, no_rnx2=False, no_rnx3=False)
Parameters:

  • statdico (dict) –

    a statdico is a dictionary associating Archives Centers to list of stations

    Exemple:
    >>> statdico['archive center 1'] = ['STA1','STA2','STA3', ...]
>>> statdico['archive center 2'] = ['STA2','STA1','STA4', ...]
    the supported archive center are (april 2024):

    • igs_cddis or igs (CDDIS data center)

    • igs_sopac (for the SOPAC/UCSD/SIO data center, but not very reliable)

    • igs_ign (IGN’s data center, main server at St Mandé)

    • igs_ign_ensg (IGN’s data center, secondary server at ENSG, Marne-la-Vallée)

    • sonel

    • euref (EPN data center hosted at ROB)

    • nav or brdc as archive center allows to download nav files (using ‘BRDC’ as station name)

    • from the ROB server, using GOP files

    • nav_rt or brdc_rt as archive center allows to download real time nav files from the BKG server

    • _not reimplemented yet_

    • rgp (IGN’s RGP St Mandé center)

    • rgp_mlv (IGN’s RGP Marne la Vallée center)

    • rgp_1Hz (IGN’s RGP, all the 24 hourly rinex for the day will be downloaded)

    • renag

    • ovsg

    • unavco

    • geoaus (Geosciences Australia)

    • ens_fr

    • _not reimplemented yet_

  • output_dir (str) – the root directory on your local drive were to store the RINEXs

  • archtype (str) –

    string describing how the archive directory is structured, e.g:

    • stat

    • stat/year

    • stat/year/doy

    • year/doy

    • year/stat

    • week/dow/stat

    • etc …

  • passwd (user &) – user & password for a secure server

  • path_ftp_crawled_files_save (str) – will save at the given path (directory+filename) in a CSV file containing the list of the existing RINEXs found on the server by the FTP crawler. It allows to use this list directly if one face a timeout during the download part.

  • path_ftp_crawled_files_load (str) – load and use the list of the existing RINEXs found on the FTP server, generated by a previous run of the FTP crawler (called by download_gnss_rinex or directly by crawl_ftp_files). a new call of crawl_ftp_files can be bypassed with skip_crawl.

  • skip_crawl (bool) – when a list of the existing RINEXs found on the FTP server is provided with path_ftp_crawled_files_load, skip a new call of crawl_ftp_files. Then, we assume the provided list as a complete one, ready for the download step

  • path_all_ftp_files_save (str) – will save at the given path (directory+filename) in a CSV file ALL the remote files found on the FTP server.

  • quiet_mode (bool) – List the available RINEXs without downloading them. Useful only if path_ftp_crawled_files_save is given

  • final_archive_for_sup_check (str) – The final archive path or a file containing the archived RINEXs in their final destination. useful if the final archive is different from archive_dir * not re-implemented yet *

  • force (bool) – Force the download even if the file already exists locally

  • no_rnx3 (no_rnx2 &) – limit the search/download to RINEX2 (short names) and/or RINEX3 (long names) depending on the boolean given

Returns:

  • url_list (list of str) – list of URLs

  • savedir_list (list of str) – list of downloaded products paths

  • Minimal exemple

  • ————— – >>> statdic = dict() >>> statdic[‘igs_cddis’] = [‘ZIMM’,’tlse’] >>> archive_dir = ‘/home/USER/test_dl_rnx’ >>> startdate = dt.datetime(2020,1,1) >>> enddate = dt.datetime(2020,1,31) >>> geodezyx.operational.download_gnss_rinex(statdic, output_dir, startdate, enddate)

geodezyx.operational.download_rinex.effective_save_dir(parent_archive_dir, stat, date, archtype='stat')

INTERNAL_FUNCTION

archtype =

stat stat/year stat/year/doy year/doy year/stat week/dow OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_rinex.ens_fr_legacy(stat, date)
geodezyx.operational.download_rinex.euref_server(stat, date)
geodezyx.operational.download_rinex.gen_crawl_table(statdico, date_range, output_dir, archtype, no_rnx2, no_rnx3)
geodezyx.operational.download_rinex.geoaus_server_legacy(stat, date)

Geosciences Australia ex : ftp://ftp.ga.gov.au/geodesy-outgoing/gnss/data/daily/2010/10063/

geodezyx.operational.download_rinex.igs_cddis_nav_server_legacy(stat, date)
geodezyx.operational.download_rinex.igs_cddis_server(stat, date)
geodezyx.operational.download_rinex.igs_ign_ensg_server(stat, date)
geodezyx.operational.download_rinex.igs_ign_server(stat, date)
geodezyx.operational.download_rinex.igs_sopac_server(stat, date)
geodezyx.operational.download_rinex.nav_bkg_server(stat, date)
geodezyx.operational.download_rinex.nav_rob_server(stat, date)
geodezyx.operational.download_rinex.orpheon_server_legacy(stat, date, user='', passwd='')
geodezyx.operational.download_rinex.ovsg_server_legacy(stat, date, user='', passwd='')
geodezyx.operational.download_rinex.renag_server_legacy(stat, date)
geodezyx.operational.download_rinex.rgp_ign_mlv_server_legacy(stat, date)
geodezyx.operational.download_rinex.rgp_ign_smn_1_hz_server_legacy(stat, date)
geodezyx.operational.download_rinex.rgp_ign_smn_server_legacy(stat, date)
geodezyx.operational.download_rinex.rnx_regex_indir(rnx_regex, dir_files_list)
geodezyx.operational.download_rinex.sonel_server(stat, date)
geodezyx.operational.download_rinex.unavco_server_legacy(stat, date)
geodezyx.operational.download_rinex.uwiseismic_server_legacy(stat, date, user='', passwd='')

geodezyx.operational.download_rinex_legacy module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_rinex_legacy.download_gnss_rinex_legacy(statdico, archive_dir, startdate, enddate, archtype='stat', parallel_download=4, sorted_mode=False, user='', passwd='', filter_ftp_crawler=True, path_ftp_crawled_files_save=None, path_ftp_crawled_files_load=None, quiet_mode=False, final_archive_for_sup_check=None, force=False)
Parameters:

statdico (dict) –

a statdico is a dictionary associating Archives Centers to list of stations

Exemple:
>>> statdico['archive center 1'] = ['STA1','STA2','STA3', ...]
>>> statdico['archive center 2'] = ['STA2','STA1','STA4', ...]
the supported archive center are (july 2015):

igs_cddis or igs (cddis center)

igs_sopac (for the sopac/ucsd/sio center, but not very reliable)

rgp (IGN’s RGP St Mandé center)

rgp_mlv (IGN’s RGP Marne la Vallée center)

rgp_1Hz (IGN’s RGP, all the 24 hourly rinex for the day will be downloaded)

renag

ovsg

unavco

sonel

geoaus (Geosciences Australia)

ens_fr

nav or brdc as archive center allows to download nav files (using ‘BRDC’ as station name) from the ROB server, using GOP files

nav_rt or brdc_rt as archive center allows to download real time nav files from the BKG server

archtypestr

string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …

sorted_modebool
if False:

using the map multiprocess fct so the download order will be scrambled

if True:

using the apply multiprocess fct so the download order will be in the chrono. order

The scrambled (False) is better, bc. it doesn’t create zombies processes

user & passwdstr

user & password for a locked server

filter_ftp_crawlerbool

use an improved FTP crawler to find which files actually exist to accelerate the download. If path_ftp_crawled_files_load is given, use this loaded list.

path_ftp_crawled_files_savestr

will save at the given path (directory+filname) in a pickle containing the list of the existing RINEXs found on the server by the FTP crawler. It allows to use this list directly if one face a timeout during the download part. NB for advanced users: the pickle is a tuple (urllist,savedirlist)

path_ftp_crawled_files_loadstr

load and use the list of the existing RINEXs found on the FTP server, generated by a previous run of the FTP crawler (called by download_gnss_rinex or directly by crawl_ftp_files). overrides an internal call of crawl_ftp_files.

quiet_modebool

List the available RINEXs without downloading them. Useful only if path_ftp_crawled_files_save is given

final_archive_for_sup_checkstr

The final archive path or a file containing the archived RINEXs in their final destination. useful if the final archive is different from output_dir

forcebool

Force the download even if the file already exists locally

Returns:

  • url_list (list of str) – list of URLs

  • savedir_list (list of str) – list of downloaded products paths

  • Minimal exemple

  • ————— – >>> statdic = dict() >>> statdic[‘igs_cddis’] = [‘ZIMM’] >>> output_dir = ‘/home/USER/test_dl_rnx’ >>> startdate = dt.datetime(2000,1,1) >>> enddate = dt.datetime(2000,1,31) >>> geodezyx.operational.download_gnss_rinex_legacy(statdic, output_dir, startdate, enddate)

geodezyx.operational.download_rinex_legacy.effective_save_dir(parent_archive_dir, stat, date, archtype='stat')

INTERNAL_FUNCTION

archtype =

stat stat/year stat/year/doy year/doy year/stat week/dow OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_rinex_legacy.ens_fr_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.geoaus_server_legacy(stat, date)

Geosciences Australia ex : ftp://ftp.ga.gov.au/geodesy-outgoing/gnss/data/daily/2010/10063/

geodezyx.operational.download_rinex_legacy.igs_cddis_nav_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.igs_cddis_server_legacy(stat, date, user='', passwd='')
geodezyx.operational.download_rinex_legacy.igs_sopac_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.multi_archiver_rinex(rinex_lis, parent_archive_dir, archtype='stat', move=True, force_mv_or_cp=False)

from rinex_lis, a list of rinex (generated by the function multi_finder_rinex)

move (if move=True) of copy (if move=False) those rinexs in the parent_archive_dir according to the archtype, string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …

geodezyx.operational.download_rinex_legacy.multi_downloader_rinex(**kwargs)
geodezyx.operational.download_rinex_legacy.nav_bkg_server(stat, date)
geodezyx.operational.download_rinex_legacy.nav_rob_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.orpheon_server_legacy(stat, date, user='', passwd='')
geodezyx.operational.download_rinex_legacy.ovsg_server_legacy(stat, date, user='', passwd='')
geodezyx.operational.download_rinex_legacy.renag_server(stat, date)
geodezyx.operational.download_rinex_legacy.rgp_ign_mlv_server(stat, date)
geodezyx.operational.download_rinex_legacy.rgp_ign_smn_1_hz_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.rgp_ign_smn_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.rnx_long2short_name(longname_filepath_in)

MUST BE IMPROVED

geodezyx.operational.download_rinex_legacy.sonel_server_legacy(stat, date)

ex : ftp://ftp.sonel.org/gps/data/2015/001/

geodezyx.operational.download_rinex_legacy.unavco_server_legacy(stat, date)
geodezyx.operational.download_rinex_legacy.uwiseismic_server(stat, date, user='', passwd='')

geodezyx.operational.download_utils module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

class geodezyx.operational.download_utils.MyFTP_TLS(host='', user='', passwd='', acct='', *, context=None, timeout=<object object>, source_address=None, encoding='utf-8')

Bases: FTP_TLS

This class is a subclass of FTP_TLS from the ftplib module. It is used to create an FTPS client that shares the TLS session. This is to avoid the error: ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2396) Source: https://stackoverflow.com/questions/14659154/ftps-with-python-ftplib-session-reuse-required

ntransfercmd(cmd, rest=None)

Initiate a data transfer over a new connection.

ntransfercmd(cmd, rest=None)

Initiate a data transfer over a new connection.

Parameters:

  • cmd (str) – The command to send to the server.

  • rest (str, optional) – A string that contains a marker representing where the server is to restart the operation’s data transfer. Default is None.

Returns:

The connection and the expected size of the data.

Return type:

tuple

Notes

If the protection level is set to private (i.e., _prot_p is True), the connection is wrapped in an SSL/TLS layer.

geodezyx.operational.download_utils.downloader(url, savedir, force=False, check_if_file_already_exists_uncompressed=True)

general function to download a file

can also handle non secure FTP

geodezyx.operational.download_utils.downloader_wrap(intup)
geodezyx.operational.download_utils.effective_save_dir_orbit(parent_archive_dir, calc_center, date, archtype='year/doy/')

INTERNAL_FUNCTION

archtype =

stat stat/year stat/year/doy year/doy year/stat week/dow wkwwww : use a GFZ’s CF-ORB wk<wwww> naming OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_utils.ftp_downld_core(ftp_obj, filename, localdir)

Performs the FTP download if we are already in the correct FTP folder. This is an internal function of ftp_downloader.

Parameters:

  • ftp_obj (FTP object) – The FTP object used to connect to the FTP server.

  • filename (str) – The name of the file to be downloaded.

  • localdir (str) – The local directory where the downloaded file should be saved.

Returns:

The local path of the downloaded file and a boolean indicating whether the download was successful.

Return type:

tuple

Notes

This function first checks if the local directory exists, if not it creates it. Then it checks if the file already exists locally, if it does, it logs a message and returns. If the file does not exist, it tries to download the file from the FTP server. If the download is successful, it logs a success message and returns the local path and True. If the download fails, it logs a failure message and returns the local path and False.

geodezyx.operational.download_utils.ftp_download_front(urls, savedirs, parallel_download=1, secure_ftp=False, user='anonymous', passwd='anonymous@isp.com', force=True)

This function is used to download files from FTP servers in parallel.

Parameters:

  • urls (str or list) – The URL or list of URLs of the files to be downloaded.

  • savedirs (str or list) – The directory or list of directories where the downloaded files should be saved.

  • parallel_download (int, optional) – The number of parallel downloads to be performed. Default is 1.

  • secure_ftp (bool, optional) – If True, uses FTPS for secure file transfer. Default is False.

  • user (str, optional) – The username for the FTP server. Default is “anonymous”.

  • passwd (str, optional) – The password for the FTP server. Default is ‘anonymous@isp.com’.

  • force (bool, optional) – If True, forces the download even if the file already exists. Default is True.

Return type:

None

Notes

This function uses the ThreadPool for parallel downloads.

geodezyx.operational.download_utils.ftp_downloader(ftp_obj, full_remote_path, localdir)

Downloads a file through FTP protocol.

Parameters:

  • ftp_obj (FTP object) – The FTP object used to connect to the FTP server.

  • full_remote_path (str) – The full path of the file on the FTP server.

  • localdir (str) – The local directory where the downloaded file should be saved.

Returns:

The output of the ftp_downloader_core function.

Return type:

tuple

Notes

This function changes the current working directory of the FTP object to the directory of the file to be downloaded, and then calls the ftp_downloader_core function to download the file.

geodezyx.operational.download_utils.ftp_downloader_wo_objects(tupin)

create the necessary FTP object

should not be used anymore

geodezyx.operational.download_utils.ftp_downloader_wrap(intup)

This function is a wrapper for the ftp_downloader function. It unpacks the input tuple and passes it to the ftp_downloader function.

Parameters:

intup (tuple) – A tuple containing the parameters to be passed to the ftp_downloader function.

Returns:

The output of the ftp_downloader function.

Return type:

tuple

geodezyx.operational.download_utils.ftp_files_crawler_legacy(urllist, savedirlist, secure_ftp)

filter urllist,savedirlist generated with download_gnss_rinex with an optimized FTP crawl

geodezyx.operational.download_utils.ftp_objt_create(secure_ftp_inp=False, host='', chdir='', parallel_download=1, user='anonymous', passwd='', retry_count=3)

This function creates and returns an FTP object and a list of FTP objects for multiple downloads.

Parameters:

  • secure_ftp_inp (bool) – If True, uses FTPS for secure file transfer. Default is False.

  • host (str, optional) – The hostname of the FTP server. Default is an empty string.

  • chdir (str, optional) – The directory to change to after connecting to the FTP server. Default is an empty string.

  • parallel_download (int, optional) – The number of parallel downloads to be performed. Default is 1.

  • user (str, optional) – The username for the FTP server. Default is “anonymous”.

  • passwd (str, optional) – The password for the FTP server. Default is an empty string.

  • retry_count (int, optional) – The number of times to retry creating the FTP object. Default is 3.

Returns:

The main FTP object for crawling and a list of FTP objects for parallel downloads.

Return type:

tuple

Notes

This function creates an FTP object using the appropriate constructor based on the secure_ftp_inp parameter. It then creates a list of FTP objects for multiple downloads. If a directory is specified, it changes the current working directory of the main FTP object to that directory.

geodezyx.operational.download_utils.start_end_date_easy(start_year, start_doy, end_year, end_doy)

generates start/end datetimes from a start/end year/day of year

Parameters:

  • start_year (int) – start year.

  • start_doy (int) – start day of year.

  • end_year (int) – end year.

  • end_doy (int) – end day of year.

Returns:

  • start (datetime) – converted start datetime.

  • end (datetime) – converted end datetime.

geodezyx.operational.groops_frontend module

Created on Tue Jan 17 10:54:49 2023

@author: psakicki

geodezyx.operational.groops_frontend.groops_basic_runner(xml_cfg_path='', global_var_dict={}, xml_var_dict={}, dry_run=False, verbosity='ERROR', log_dir=None, groops_bin_path='/opt/softs_gnss/groops/bin/groops')
Parameters:

  • xml_cfg_path (str, optional) – the XML config file for GROOPS. The default is “”.

  • global_var_dict (dict, optional) – A dictionnary to change the global variables values, like: global_var_dict[“global_var_name”] = new_value The default is dict().

  • xml_var_dict (dict, optional) – A dictionnary to change the values in the config XML file. Not implemented yet. The default is dict().

  • dry_run (bool, optional) – If True print the command but do not run it . The default is False.

  • verbosity (str, optional) – verbosity level of the console logger use keywords CRITICAL, ERROR, WARNING, INFO, DEBUG The default is ‘ERROR’.

  • log_dir (str, optional) – If provided, directory where the logs are stored. The default is None.

  • groops_bin_path (TYPE, optional) – Path of the GROOPS bin. The default is ‘/opt/softs_gnss/groops/bin/groops’.

Return type:

None.

geodezyx.operational.groops_frontend.groops_ppp_full_runner(rinex_path, project_name, igs_ac_10char, cfg_files_dict, log_root_dir, vmf_tropo_root_dir, prods_gnss_root_dir, cfg_files_root_dir, sitelogs_root_dir, groops_bin_path='/opt/softs_gnss/groops/bin/groops', verbosity='ERROR')

High level function to run a GROOPS’s PPP job This function download IGS’s products, convert them, and run the PPP job see the Notes below for more details

Parameters:

  • rinex_path (str) – the path of the RINEX file to process.

  • project_name (str) – a personalized name for your processing project

  • igs_ac_10char (str) – the 10 char. ID for the IGS AC you want to use e.g. ‘COD0OPSFIN’ can handle operational (OPS) and MGEX (MGX) lines

  • cfg_files_dict (dict) – a dictionary controlling the conversion/processing steps and the corresponding config files.

  • log_root_dir (str) – directory path where the frontend logs will be written

  • vmf_tropo_root_dir (str) – directory path where the VMF3/ECMWF grids will be stored (auto download)

  • prods_gnss_root_dir (str) – directory path where the IGS products (not converted) will be stored (auto download)

  • cfg_files_root_dir (str) – directory path where the config files are stored.

  • sitelogs_root_dir (str) – directory path where the sitelogs files are stored.

  • groops_bin_path (str, optional) – Path of the GROOPS bin. The default is ‘/opt/softs_gnss/groops/bin/groops’.

  • verbosity (str, optional) – verbosity level of the console logger for the GROOPS’s messages use keywords CRITICAL, ERROR, WARNING, INFO, DEBUG The default is ‘ERROR’. This verbosity level does not apply to this function

Note

the config files must be checked and edited manually to fit your environnement config it is the config files which contains the most useful parameters use groopsGui to help you

prototype for config files are in: …/geodezyx/000_exemples/groops_frontend/configfiles/

do not forget to update on a regular basis GROOPS’s data folder: ` https://ftp.tugraz.at/outgoing/ITSG/groops/data.zip`

Return type:

None.

geodezyx.operational.groops_frontend.log_subprocess(pipe, logger=None, file=None, file2=None)

Intern fuction for subprocess_frontend2 to write the stdout/err in the console logger + a logfile

geodezyx.operational.groops_frontend.subprocess_frontend2(cmd_in, save_log=True, log_dir=None, log_name_out='out.log', log_name_err='err.log', logname_timestamp=True, err_also_in_outfile=True, logger_objt_level_out=None, logger_objt_level_err=None)

A generic frontend to run an extern command through subprocess and write the stdout and stderr outputs in log files.

Parameters:

  • cmd_in (str) – The subprocess command.

  • save_log (str, optional) – export as log the stdout/stderr in files. The default is True.

  • log_dir (str, optional) – directory where the logs will be stored. The default is None.

  • log_name_out (str, optional) – filename of the stdout log. The default is “out.log”.

  • log_name_err (str, optional) – filename of the stderr log. The default is “err.log”.

  • logname_timestamp (str, optional) – add a timestamp as prefix. The default is True.

  • logger_objt_level_out (method of a Logger object, optional) – set the logger level of the stdout messages. can be logger.info or logger.debug for instance. The default is None (logger.info per default then).

  • logger_objt_level_err (method of a Logger object, optional) – set the logger level of the stderr messages. can be logger.error or logger.critical for instance. The default is None (logger.error per default then).

Returns:

exitcode – the exit code of the command.

Return type:

int

Notes

Inspired by: https://stackoverflow.com/questions/21953835/run-subprocess-and-print-output-to-logging And for the threading: https://stackoverflow.com/questions/6809590/merging-a-python-scripts-subprocess-stdout-and-stderr-while-keeping-them-disti

geodezyx.operational.groops_frontend.vmf_tropo_downloader(output_dir, startdate=datetime.datetime(2019, 1, 1, 0, 0), enddate=datetime.datetime(2019, 1, 1, 0, 0), model='VMF3', version='OP')

geodezyx.operational.hector_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the time series velocities estimation software HECTOR.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.hector_frontend.MJD2dt(mjd_in)
geodezyx.operational.hector_frontend.get_FLH_from_NEUfile(neufilepath)
geodezyx.operational.hector_frontend.keeping_specific_stats(listoffiles, specific_stats, invert=False)

if invert = True : NOT keeping BUT removing specific stats

geodezyx.operational.hector_frontend.momfile_trend_processing(inp_momfile, generik_conf_file, outdir='', remove_ctl_file=True)
geodezyx.operational.hector_frontend.multi_momfile_trend_processing(inpdir, generik_conf_file, outdir='', extention='pre.mom', remove_ctl_file=True, specific_stats=(), invert_specific=False)
geodezyx.operational.hector_frontend.multi_neufile_outlier_removing(inpdir, generik_conf_file, outdir='', extention='neu', specific_stats=(), invert_specific=False, remove_ctl_file=True)
geodezyx.operational.hector_frontend.multi_sumfiles_trend_extract(inp_dir, out_dir, out_prefix, raw_neu_dir='', specific_stats=(), invert_specific=False, style='epc')

style = globk OR epc make a GLOBK style .vel file or make a dirty velocity file compatible with EPC

geodezyx.operational.hector_frontend.neufile_outlier_removing(inp_neufile, generik_conf_file, outdir='', remove_ctl_file=True)

from a NEU file => removeoutlier preprocessing => 3 MOM files (one per component)

geodezyx.operational.hector_frontend.sumfiles_to_statdico(inpdir, specific_stats=(), invert_specific=False)

this fct search for every sum file in a folder a stat dico contains no data only the paths to the E,N,U sum files statdico[stat] = [path/E.sum,path/N.sum,path/U.sum ]

for each stat getting the 3 ENU sum files

Thoses lists will be send in sumfiles_trend_extract

geodezyx.operational.hector_frontend.sumfiles_trend_extract(listof3ENUsumfile)
geodezyx.operational.hector_frontend.velfile_from_a_list_of_statVsV_tuple(listoftup, out_dir, out_prefix, raw_neu_dir='', style='epc')

geodezyx.operational.midas_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the time series velocities estimation software MIDAS.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.midas_frontend.midas_plot(path_tenu, path_vel='', path_step='')

based on a plot for TimeSeriePoint

geodezyx.operational.midas_frontend.midas_run(tenu_file_path, work_dir='', path_midas_soft='', step_file_path='', with_plot=True, keep_plot_open=True)
geodezyx.operational.midas_frontend.midas_vel_files_2_pandas_DF(vel_files_in)

Convert MIDAS Velocity files to a Pandas DataFrame

Parameters:

vel_files_in (str or list of str) – if list of str, will consider directly the files inside the list if str, can be the path of a single file, or a generic (wilcard) path to consider several files

Returns:

DF

Return type:

Pandas DataFrame

geodezyx.operational.pride_pppar_frontend module

Created on Wed Feb 14 18:01:49 2024

@author: psakic

geodezyx.operational.pride_pppar_frontend.dl_brdc_pride_pppar(prod_parent_dir, date_list)

Downloads BRDC (Broadcast Ephemeris) files for PRIDE PPPAR from a given directory and date list.

Parameters:

  • prod_parent_dir (str) – The parent directory where the products are stored.

  • date_list (iterable of datetime) – The list of dates for which the BRDC files are to be downloaded.

Returns:

A list of downloaded BRDC files.

Return type:

list

Notes

This function first rounds the dates in the date list to the nearest day and removes duplicates. It then downloads the BRDC files for each unique date using the operational.download_gnss_rinex function. The downloaded files are appended to a list which is returned at the end.

geodezyx.operational.pride_pppar_frontend.dl_prods_pride_pppar(prod_parent_dir, date_list, prod_ac_name)

Downloads GNSS products for PRIDE PPPAR from a given directory and date list.

Parameters:

  • prod_parent_dir (str) – The parent directory where the products are stored.

  • date_list (iterable of datetime) – The list of dates for which the GNSS products are to be downloaded.

  • prod_ac_name (str) – The name of the analysis center providing the products.

Returns:

A list of downloaded GNSS products.

Return type:

list

Notes

This function downloads various GNSS products such as orbits, clocks, biases, etc. It iterates over specified data centers and attempts to download the products. If at least 5 products are found, the function stops further downloads.

geodezyx.operational.pride_pppar_frontend.get_best_latency(prod_lis_inp)

Selects the best latency from a list of product files.

Internal function for get_right_prod.

Parameters:

prod_lis_inp (list) – A list of product file paths.

Returns:

The best latency found in the list. But can be several if the latency is the same for several products.

Return type:

list

geodezyx.operational.pride_pppar_frontend.get_right_brdc(brdc_lis_inp, tmp_dir_inp)

Selects the appropriate BRDC (Broadcast Ephemeris) file from a list and unzips it.

Parameters:

  • brdc_lis_inp (list) – A list of BRDC file paths.

  • tmp_dir_inp (str) – The directory where the unzipped BRDC file will be stored.

Returns:

A tuple containing the original BRDC file path and the unzipped BRDC file path.

Return type:

tuple

geodezyx.operational.pride_pppar_frontend.get_right_prod(prod_lis_inp, tmp_dir_inp, prod_name, default_fallback)

Selects the appropriate product file from a list and unzips it if necessary.

Parameters:

  • prod_lis_inp (list) – A list of product file paths.

  • tmp_dir_inp (str) – The directory where the unzipped product file will be stored.

  • prod_name (str) – The name of the product.

  • default_fallback (bool) – If True, uses default values if products are not found.

Returns:

A tuple containing the unzipped product file path and the original product file path.

Return type:

tuple

geodezyx.operational.pride_pppar_frontend.pride_pppar_mp_wrap(kwargs_inp)
geodezyx.operational.pride_pppar_frontend.pride_pppar_runner(rnx_path_list, cfg_template_path, prod_ac_name, prod_parent_dir, tmp_dir, cfg_dir, run_dir, multi_process=1, cfg_prefix='pride_pppar_cfg_1a', mode='K', options_dic={}, bin_dir=None, force=False, dl_prods=False, default_fallback=False, dl_prods_only=False, clean_run_dir=True)
geodezyx.operational.pride_pppar_frontend.pride_pppar_runner_mono(rnx_path, cfg_template_path, prod_ac_name, prod_parent_dir, tmp_dir, cfg_dir, run_dir, cfg_prefix='pride_pppar_cfg_1a', mode='K', options_dic={}, bin_dir=None, force=False, dl_prods=False, default_fallback=False, dl_prods_only=False, clean_run_dir=True)

Runs the PRIDE PPPAR process for a single RINEX file.

Parameters:

  • rnx_path (str) – The path to the RINEX file.

  • cfg_template_path (str) – The path to the configuration template file.

  • prod_ac_name (str) – The name of the analysis center providing the products.

  • prod_parent_dir (str) – The parent directory where the products are stored.

  • tmp_dir (str) – The temporary directory for intermediate files.

  • cfg_dir (str) – The directory for configuration files.

  • run_dir (str) – The directory where the run results will be stored.

  • cfg_prefix (str, optional) – The prefix for the configuration file name. Default is “pride_pppar_cfg_1a”.

  • mode (str, optional) – The mode for the PRIDE PPPAR process. Default is “K”.

  • options_dic (dict, optional) – Additional options for the PRIDE PPPAR process. Default is an empty dictionary.

  • bin_dir (str, optional) – The directory where the PRIDE PPPAR binaries are located. Default is None.

  • force (bool, optional) – If True, forces the process to run even if logs already exist. Default is False.

  • dl_prods (bool, optional) – If True, downloads the necessary products. Default is False.

  • default_fallback (bool, optional) – If True, uses default values if products are not found. Default is False.

  • dl_prods_only (bool, optional) – If True, only downloads the products and exits. Default is False.

  • clean_run_dir (bool, optional) – If True, removes temporary files inside the run directory. Default is True.

Return type:

None

geodezyx.operational.pride_pppar_frontend.remove_regex_reserved_characters(input_string)

Removes all REGEX reserved characters from the input string.

Parameters:

input_string (str) – The string to be cleaned of REGEX reserved characters.

Returns:

The cleaned string with all REGEX reserved characters removed.

Return type:

str

geodezyx.operational.pride_pppar_frontend.run_command(command)

Runs a shell command and captures both stdout and stderr.

Parameters:

command (str) – The shell command to be executed.

Notes

This function uses subprocess.Popen to run the command in a new process. It continuously reads and prints stdout and stderr until the process finishes. The function prints the return code of the process once it completes.

geodezyx.operational.rinex_lister_plotter module

@author: psakic

This sub-module of geodezyx.operational generates RINEX timeline plot.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.rinex_lister_plotter.listing_gins_timeline(path, stat_strt, stat_end, date_strt, date_end, suffix_regex='')

find all gins listings in a folder and his subfolders and plot timeline of the avaiable listings

stat_strt,stat_end,date_strt,stat_end : where to find in the name the statname and the date

geodezyx.operational.rinex_lister_plotter.rinex_check_epochs_availability(rinex_path_list)

Gives simple statistics about RINEX avaiability for all the listed stations

Parameters:

rinex_path_list (list) – A list of rinex paths

Returns:

T – a table with statistics.

Return type:

str

geodezyx.operational.rinex_lister_plotter.rinex_lister(path, add_long_names=True)

find all rinex in a folder and his subfolders path can be a string or a tuple of string => manage multi paths :)

is very similar with softs_runner.multi_finder_rinex, gins_runner.get_rinex_list and operational.rinex_finder

Parameters:

  • path (str) – archive path.

  • add_long_names (bool, optional) – consider new names. The default is True.

Returns:

rinexfilelist – list of rinex files.

Return type:

list

Notes

operational.rinex_finder must be used in priority !!! (July 2022)

geodezyx.operational.rinex_lister_plotter.rinex_timeline(inputlist_or_paths, start=datetime.datetime(1980, 1, 1, 0, 0), end=datetime.datetime(2099, 1, 1, 0, 0), use_rinex_lister=True, dots_plot=False, jul_date_plot=False, return_figure=False, xlim_start_end=False)

Frontend function to plot the aviable RINEXs

Parameters:

  • inputlist_or_paths (iterable) – list of rinex file paths or list of archive directories.

  • start (datetime, optional) – start of the plot time span. The default is dt.datetime(1980,1,1).

  • end (datetime, optional) – end of the plot time span. The default is dt.datetime(2099,1,1).

  • use_rinex_lister (bool, optional) – if True, inputlist_or_paths is a list of archive directories. if False, inputlist_or_paths is a list of rinex file paths. The default is True.

  • dots_plot (bool, optional) – print dots instead of line. This mode should be avoided because it is slower. The default is False.

  • jul_date_plot (bool, optional) – Plot the date in julian dates. The default is False.

  • return_figure (bool, optional) – returns figure. The default is False.

  • xlim_start_end (bool, optional) – Force start and end values to be the x axis limit The default is False.

Returns:

  • datadico (dict) – datadico.

  • fig (matplotlib figure) – figure

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico(inputlist_or_paths, use_rinex_lister=True, optional_info='')

Generate a RINEX datadico

Parameters:

  • inputlist_or_paths (iterable) – list of rinex file paths or list of archive directories.

  • use_rinex_lister (bool, optional) – if True, inputlist_or_paths is a list of archive directories. if False, inputlist_or_paths is a list of rinex file paths. The default is True.

  • optional_info (str, optional) – A addtional information for the rinexs found. Usually it is the archive name The default is ‘’.

Returns:

datadico – datadico[stat] = [(rinexname1,optional1,date1) … (rinexnameN,optionalN,dateN)]

Return type:

dict

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico_merge(datadico_list, priority_list=None)

Merge different RINEXs datadico, produced by rinex_timeline_datadico coming from different archives

Parameters:

  • datadico_list (list of dict) – list of RINEX datadico.

  • priority_list (list, optional) – priority list of ‘optional_info’ (archive ID) it will erase optional_info of lower priority NB : it is not very useful, just sort datadico_list in the right order …. The default is None.

Returns:

datadico_out – a merged datadico

Return type:

dict

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico_merge_not_very_smart(datadico_list, priority_list)

Merge different RINEXs datadico, produced by rinex_timeline_datadico coming from different archives Args :

rinex_timeline_datadico : list of RINEX datadico priority_list : priority list of ‘optional_info’ (archive ID)

it will erase optional_info of lower priority

Returns :

datadico_out : a merged datadico

geodezyx.operational.rinex_lister_plotter.timeline_plotter(datadico, start=datetime.datetime(1980, 1, 1, 0, 0), end=datetime.datetime(2099, 1, 1, 0, 0), dots_plot=False, jul_date_plot=False, datadico_anex_list=[], use_only_stats_of_main_datadico=False, colordico_for_main_datadico=None, xlim_start_end=False, stats_only_list=[])
Parameters:

  • datadico (dict) – a RINEX datadico (see rinex_timeline_datadico for details).

  • start (datetime, optional) – start of the plot time span. The default is dt.datetime(1980,1,1).

  • end (datetime, optional) – end of the plot time span. The default is dt.datetime(2099,1,1).

  • dots_plot (TYPE, optional) – print dots instead of line. This mode should be avoided because it is slower. The default is False.

  • jul_date_plot (bool, optional) – Plot the date in julian dates. The default is False.

  • datadico_anex_list (list, optional) – A list of secondary datadicos. The default is [].

  • use_only_stats_of_main_datadico (bool, optional) – Use only stats of main datadico. Advanced usage. The default is False.

  • colordico_for_main_datadico (dict, optional) – Color for the main datadico. Advanced usage. The default is None.

  • xlim_start_end (bool, optional) – Force start and end values to be the x axis limit The default is False.

  • stats_only_list (list, optional) – If given, keep only the stations of this list The default is [].

Returns:

fig – figure

Return type:

matplotlib figure

Notes

A simpler version has been commited to geodezyx toolbox GitHub for archive on 20180118 15:59A

geodezyx.operational.rinex_utils module

@author: psakic

This sub-module of geodezyx.operational contains functions to manipulate RINEX files (version 2, quite obsolate).

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.rinex_utils.check_if_compressed_rinex(rinex_path)
geodezyx.operational.rinex_utils.crz2rnx(rinex_path, outdir='', force=True, path_of_crz2rnx='CRZ2RNX', verbose=True)

assuming that CRZ2RNX is in the system PATH per default

geodezyx.operational.rinex_utils.read_rnx_epoch_line(line, rnx2=True)

read the epoch line of a RINEX (2 or 3) and extract it as a datetime

Parameters:

  • line (string) – input line, assuming it’s a RINEX epoch line.

  • rnx2 (TYPE, optional) – True if RINEX2, False if RINEX3/4. The default is True.

Returns:

epochdt – The epoch of the line.

Return type:

datetime

geodezyx.operational.rinex_utils.rinex_lists_diff(rnx_lis1, rnx_lis2, out_dir=None, out_name=None, site9_col=False)

Compare two lists of RINEX files and identify differences.

Parameters:

  • rnx_lis1 (list) – First list of RINEX file paths.

  • rnx_lis2 (list) – Second list of RINEX file paths.

  • out_dir (str, optional) – Directory to save the output files. If None, no files are saved.

  • out_name (str, optional) – Base name for the output files. Defaults to ‘rnx_diff’ if not provided.

  • site9_col (bool, optional) – If True, use ‘site9’ as the column name for site identification. Defaults to False.

Returns:

  • diff1m2 (pandas.DataFrame) – DataFrame containing entries in rnx_lis1 but not in rnx_lis2.

  • diff2m1 (pandas.DataFrame) – DataFrame containing entries in rnx_lis2 but not in rnx_lis1.

  • intrsec (pandas.DataFrame) – DataFrame containing entries common to both rnx_lis1 and rnx_lis2.

  • symdiff (pandas.DataFrame) – DataFrame containing entries that are in either rnx_lis1 or rnx_lis2 but not in both.

geodezyx.operational.rinex_utils.rinex_read_epoch(input_rinex_path_or_string, interval_out=False, add_tzinfo=False, out_array=True, out_index=False)

Read the epochs contained in a RINEX File. Can handle RINEX 2 and 3

Parameters:

  • input_rinex_path_or_string (see below) – input RINEX. can be the path of a RINEX file as string or as Path object, or directly the RINEX content as a string, bytes, StringIO object or a list of lines

  • interval_out (bool, optional) – output also the intervals. The default is False.

  • add_tzinfo (bool, optional) – add timezone information in the datetime’s Epoches. The default is False.

  • out_array (bool, optional) – output results as array. The default is True.

  • out_index (bool, optional) – output also the index of the epoch line. The default is False.

Returns:

the epochs in the RINEX.

Return type:

array or list

geodezyx.operational.rinex_utils.rinex_renamer(input_rinex_path, output_directory, stat_out_name='', remove=False)
geodezyx.operational.rinex_utils.rinex_sats_checker(p_rnx)

Check the consistency of a RINEX’s sat list for each epoch

Designed for checking the consistency of IPGP-OVPF corrupted RINEXs RINEX-2 only

Parameters:

p_rnx (str) – input RINEX path.

Return type:

bad RINEX path or None.

geodezyx.operational.rinex_utils.rinex_session_id(first_epoch, last_epoch, full_mode=False)
full_mode:

gives the letter of the starting session & the length in hour of the session

geodezyx.operational.rinex_utils.rinex_spliter(input_rinex_path, output_directory, stat_out_name='', interval_size=24, compress=False, shift=0, inclusive=False, teqc_cmd='teqc')

if shift != 0: the start/end of a session is shifted of shift minutes

inclusive: delta of exaclty interval_size => add the 1st epoch of the next sess not inclusive: delta of interval_size - 1s

geodezyx.operational.rinex_utils.rinex_spliter_gfzrnx(input_rinex_path, output_directory, stat_out_name='', interval_size=86400, shift=0, inclusive=False, gfzrnx_cmd='GFZRNX', output_name='::RX3::', custom_cmds='')
geodezyx.operational.rinex_utils.rinex_start_end(input_rinex_path, interval_out=False, add_tzinfo=False, verbose=True, safety_mode=True)

Return the first and the last epoch of a RINEX file (based on the actual content of the file, not the header)

Can handle RINEX 2 and 3

Parameters:

  • input_rinex_path (TYPE) – path of the rinex file. can be the path of a RINEX or directly the RINEX content as a string

  • interval_out (bool, optional) – output also the intervals. The default is False.

  • add_tzinfo (bool, optional) – add timezone information in the datetime’s Epoches. The default is False.

  • verbose (bool, optional) – verbose mode. The default is True.

  • safety_mode (TYPE, optional) – if the epoch reading fails (e.g. in case of a compressed RINEX) activate a reading of the header and the file name as backup. The default is True.

Returns:

First, las epoches and interval if asked.

Return type:

first_epoch , last_epoch , [interval]

geodezyx.operational.rinex_utils.rinex_table_from_list(rnxs_inp, site9_col=False, round_date=False, path_col=True, size_col=False, sort=True)

From a simple RINEX list, summarize the data in an ad-hoc DataFrame.

Parameters:

  • rnxs_inp (iterable or str) – If iterable, a list of RINEX files. If str, path of an input RINEX list as text file.

  • site9_col (bool, optional) – If True, include a ‘site9’ column in the DataFrame. Defaults to False.

  • round_date (bool, optional) – If True, round the dates to the nearest day. Defaults to False.

  • path_col (bool, optional) – If True, include the ‘path’ column in the DataFrame. Defaults to True.

  • size_col (bool, optional) – If True, include a ‘size’ column in the DataFrame. Corresponds to the size of the file in bytes. It can slow down the process if the list is long. Defaults to False.

  • sort (bool, optional) –

    If True, sort the DataFrame by ‘site4’ or ‘site9’ (if site9_col is True) and ‘date’.

    Defaults to True.

Returns:

df – A DataFrame with the RINEX info in it.

Return type:

pandas.DataFrame

geodezyx.operational.rinex_utils.rnx2crz(rinex_path, outdir='', force=True, path_of_rnx2crz='RNX2CRZ')

assuming that RNX2CRZ is in the system PATH per default

geodezyx.operational.rinex_utils.same_day_rinex_check(rinex1, rinex2)
geodezyx.operational.rinex_utils.teqc_qc(rinex_path, quick_mode=False, optional_args='')

quick mode : reduced qc and no summary file written

geodezyx.operational.rtklib_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the GNSS processing software RTKLIB.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.rtklib_frontend.read_conf_file(filein)
geodezyx.operational.rtklib_frontend.rtklib_run_from_rinex(rnx_rover, rnx_base, generik_conf, working_dir, experience_prefix='', rover_auto_conf=False, base_auto_conf=True, XYZbase=[0, 0, 0], outtype='auto', calc_center='igs')
auto_conf :

read the header of the rinex and write the conf. file according to it if the mode is disabled, the antenna/rec part of the conf. file will be the same as the generic one

NB : RTKLIB “core” have it’s own reading header option. Thus, my advice is to disable the auto mode for the rover and leave ant1-postype=rinexhead in the generic conf file and enable it for the base with a XYZbase vector initialized or the good XYZ in header on the rinex (the XYZbase vector is prioritary over the base RINEX header)

(but in anycase, prepars good rinex’s headers ;) )

outtype :

‘auto’ (as defined in the generic config file) or ‘dms’ ‘deg’ ‘xyz’ ‘enu’ can manage the upper case XYZ or FLH

geodezyx.operational.track_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the GNSS processing software TRACK.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.track_frontend.run_track(temp_dir, exp_full_name, out_conf_fil, date, rnx_rover)
geodezyx.operational.track_frontend.track_runner(rnx_rover, rnx_base, working_dir, experience_prefix, XYZbase=[], XYZrover=[], outtype='XYZ', mode='short', interval=None, antmodfile='~/gg/tables/antmod.dat', calc_center='igs', forced_sp3_path='', const='G', silent=False, rinex_full_path=False, run_on_gfz_cluster=False, forced_iono_path='')