geodezyx.operational package

Submodules

geodezyx.operational.anubis_frontend module

Created on Tue Jul 5 15:48:26 2022

@author: psakic

geodezyx.operational.anubis_frontend.anubis_runner(rnx_inp, out_dir_main, xml_config_generic, period=None, interval=None, dry_run=False, download_nav=True, download_sp3=True, force=False, anubis_path='/opt/gnss_softs/bin/anubis')

Run an Anubis quality check. Designed for Anubis 3.3. Works with Anubis 3.7 at least.

Parameters:

rnx_inp (str or list) –
Input RINEXs. can be a RINEX path list, or the path of the parent archive directory

Note that this function is optimized for the batch processing of several RINEXs. If you want to process a single RINEX, gives its path in a list ([rnx_inp])
out_dir_main (str) – output main directory.
xml_config_generic (str) – path of the generic XML configuration file. will be stored in <out_dir_main>/inp See note bellow to find some exemples
period (None or int) – nominal file period in the RINEX (in hours) if None is given, guess based on the RINEX name The default is None.
interval (None or int) – nominal data interval in the RINEX (in sec) if None is given, guess based on the RINEX name The default is None.
dry_run (bool, optional) – if True, do not run the Anubis QC. Anubis QC results are stored in <out_dir_main>/out The default is False.
download_nav (bool, optional) – Download automatically the Broadcast navigation files. will be stored in <out_dir_main>/nav The default is True.
download_sp3 (bool, optional) – Download automatically the SP3 orbit files. CODE’s MGEX are used per default. will be stored in <out_dir_main>/nav The default is True.
force (bool, optional) – Per default, skip anubis execution if a xtr file already exists in out_dir_main The default is False.
anubis_path (str, optional) – path of the Anubis executable. The default is “/opt/gnss_softs/bin/anubis”

Returns:

xml_cfg_list – list of the generated XML config files.

Return type:

list

Note

Generic Configuration files for Anubis version 2 and 3 can be found here:

<...>/geodezyx/000_exemples/anubis_configfiles

or directly on the GeodeZYX’s toolbox GitHub repository:

https://github.com/GeodeZYX/geodezyx-toolbox/tree/master/geodezyx/000_exemples/anubis_configfiles

geodezyx.operational.cluster_gfz module

@author: psakic

This sub-module of geodezyx.operational contains functions to send runs to the GFZ’s cluster in a batch mode.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.cluster_gfz.cluster_GFZ_run(commands_list, bunch_on_off=True, bunch_job_nbr=10, bunch_wait_time=600, bj_check_on_off=True, bj_check_mini_nbr=2, bj_check_wait_time=120, bj_check_user='auto', add_cjob_cmd_prefix=True, wait_sleeping_before_launch=5)

Parameters:

commands_list (list of str) – List of commands you want to run (one command per job).
bunch_on_off (bool, optional) – If False, send all the jobs to the cluster at once. If True, just send <bunch_job_nbr> of them. See the options below. The default is True.
bunch_job_nbr (int, optional) – number of jobs which will be sent to the cluser at once. The default is 10.
bunch_wait_time (int, optional) – Minimal time between two bunch runs in sec. The default is 600.
bj_check_on_off (bool, optional) – Do a check before a new bunch run, if previous jobs are still running. The default is True.
bj_check_mini_nbr (int, optional) – The fuctions will wait <bj_check_wait_time> sec more if <bj_check_mini_nbr> or more jobs are running. The default is 2.
bj_check_wait_time (int, optional) – wait time between two checks in sec. The default is 120.
bj_check_user (str, optional) – username you want to check in the batchjobs. The default is “auto” i.e. your own username.
add_cjob_cmd_prefix (bool, optional) – If, the input commands in command_list do not contain the cjob prefix command, it will be added automatically The default is True.
wait_sleeping_before_launch (int, optional) – Waiting time between two successive runs. (To cancel the run if necessary) The default is 5.

Return type:

None.

geodezyx.operational.cluster_gfz.number_job_user(bj_check_user=None, verbose=True): Internal function for sleep_job_user

geodezyx.operational.cluster_gfz.sleep_job_user(bj_check_user=None, minjob=20, bj_check_wait_time=20): Internal function for cluster_GFZ_run

geodezyx.operational.download_cddis module

@author: mansur This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_cddis.download_rp3_cddis(week_ini=None, ac=None, fmt=None, out_path=None)

Download sp3 weekly files for repro3 products, from cddis (NASA). Note: For the moment is only for sp3 and clk.

Parameters:

week_ini (GPS week for download) –
fmt (searching format file e.g. sp3) –
ac (name of the anlisys center) –
out_path (path where the products will be download) –

Return type:

No returns.

geodezyx.operational.download_cddis.list_daily_files_ftp(ftp_dir_list=None, fmt='sp3', ac='COD', week=1900, dayw=0)

List files inside one directory in a ftp serve. Note that this uses an anonymous user.

Parameters:

ftp_dir_list (list with the files in the ftp (comming from the previous fct: list_file_in_ftp)) –
fmt (searching format file e.g. sp3) –
ac (name of the anlisys center) –
week (gps week) –
day (day of the week) –

Return type:

Return a list with the the information about the directory.

geodezyx.operational.download_cddis.list_file_in_ftp(email='mansur@gfz-potsdam.de', host='gdc.cddis.eosdis.nasa.gov', directory='gnss/products/repro3/1900')

List files inside one directory in a ftp serve. Note that this uses an anonymous user.

Parameters:

email (login email) –
host (ftp server) –
directory (directory inside the server) –

Return type:

Return a list with the the information about the directory.

geodezyx.operational.download_dropbox module

Created on Thu Feb 2 14:37:57 2023

@author: psakic

Based on https://practicaldatascience.co.uk/data-science/how-to-use-the-dropbox-api-with-python

The Dropbox app parameters are here https://www.dropbox.com/developers

The important permission parameters are:: sharing.write the other can be read only

reset the token after changing the permissions!

geodezyx.operational.download_dropbox.dropbox_connect(dropbox_access_token): Create a connection to Dropbox.

geodezyx.operational.download_dropbox.dropbox_list_files(dbx, path)

Parameters:

dbx (dropbox.dropbox_client.Dropbox) – The dropbox_connect output object.
path (str) – the path of your Dropbox folder. (the root folder with your name not included) e.g. dropbox_path = “/split_2021_c/” dropbox_path = “/TE/RINEX 3.04/” + str(year) dropbox_path = “/RINEX 3.04/” + str(year)

Returns:

df (DataFrame) – a Pandas dataframe of files in a given Dropbox folder path in the Apps directory.
files (list of dropbox.files.FileMetadata) – list of dropbox.files.FileMetadata.

geodezyx.operational.download_dropbox.dropbox_wget_cmds(DFfiles, dbx, out_dir_dwnld_files, out_dir_cmd_list, cmd_list_suffix, run_get_temp_link=False, run_get_perma_link=True, teria_archive=True, start_date=datetime.datetime(1980, 1, 1, 0, 0), end_date=datetime.datetime(2099, 1, 1, 0, 0))

Generate URL and wget commands to download files from a dropbox Especially to manage Teria’s RINEXs but no only.

Parameters:

DFfiles (DataFrame) – Files DataFrame outputed from dropbox_list_files.
dbx (dropbox.dropbox_client.Dropbox) – The dropbox_connect output object.
out_dir_dwnld_files (str) – directory for the future downloaded files.
out_dir_cmd_list (str) – directory for the URL list/wget commands.
cmd_list_suffix (str) – suffix for the URL list/wget commands files.
run_get_temp_link (bool, optional) – request temporary link. The default is False.
run_get_perma_link (bool, optional) – request permanent link. Recommeded. Overrides run_get_temp_link. The default is True.
teria_archive (bool, optional) – manage time for the Teria archive. The default is True.
start_date (datetime, optional) – start date. The default is dt.datetime(1980,1,1).
end_date (datetime, optional) – end date. The default is dt.datetime(2099,1,1).

Returns:

WgetList (list) – list of wget commands.
UrlDF (DataFrame) – DataFrame of the download URLs.

geodezyx.operational.download_find_files module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_find_files.find_IGS_products_files(parent_dir, File_type, ACs, date_start, date_end=None, recursive_search=True, severe=True, compressed='incl', regex_old_naming=True, regex_new_naming=True, regex_igs_tfcc_naming=True, add_weekly_file=False, add_hourly_file=False)

Find all product files in a parent folder which correspond to file type(s), AC(s) and date(s)

Parameters:: parent_dir (str or list of str) – The parent directory (i.e. the archive) where files are stored can be a string (path of the archive) or a list of file paths (given by the function utils.find_recursive) in order to gain time

File_typestr or list of str: File type(s) researched (sp3, erp, clk …) can be a list of string for several file type paths or a string like ‘sp3’ if only one file type is researched
ACs‘all’ or str or list of str: AC(s) researched can be a list of string for several ACs or a string like ‘gfz’ if only one AC is researched if ‘all’, search for all the ACs
date_startdt.datetime or 2-tuple list of int: begining of the time period researched can be a datetime or a 2-tuple (wwww,d) e.g. (1990,0)
date_endNone or dt.datetime or 2-tuple list of int: end of the time period researched can be a datetime or a 2-tuple (wwww,d) e.g. (1990,0) if None, then only date_start is researched
severebool: If True, raises an exception if something goes wrong
compressedstr: How the compressed files are handled “incl”: include the compressed files “only”: only consider the compressed files “excl”: exclude the compressed files
regex_old_namingbool: Handle old naming format
regex_new_namingbool: Handle new naming format
regex_igs_tfcc_namingbool: Handle TFCC specific format (for SINEX files)
add_weekly_filebool: Also handle the weekly file (day 7) Implemented only for the old naming format (for the moment)
add_hourly_filebool: Also handle ultra rapide hourly file Implemented only for the new naming format

Returns:: Files_select_cumul_list – List of files found
Return type:: list

geodezyx.operational.download_find_files.multi_finder_rinex(main_dir, rinex_types=('o', 'd', 'd.Z', 'd.z'), specific_stats=[])

from a main_dir, find all the rinexs in this folder and his subfolder (corresponding to the rinex_types) and return a list of the found rinexs

Designed for RINEX-2 / short names only

is very similar with geodetik.rinex_lister, gins_runner.get_rinex_list, operational.rinex_finder

operational.rinex_finder must be used in priority !!! (July 2022)

geodezyx.operational.download_find_files.read_rinex_list_table(rnx_list_inp)

Generate a Table from a RINEX list

Parameters:: p (str, list or iterable) – RINEX list. Can be a Python iterable (a list) or a path to a list file (a string)
Returns:: DF – a RINEX Table.
Return type:: DataFrame

Note

From script …/geodezyx_toolbox_PS_perso_scripts/IPGP_OVS/rinex_lister/rinex_list_2019_2022_mk01.py

geodezyx.operational.download_find_files.rinex_finder(main_dir, short_name=True, long_name=True, gfz_godc_name=True, compressed=None, specific_sites=[], start_epoch=None, end_epoch=None)

Parameters:

main_dir (str) – main directory where the RINEXs are stored.
short_name (bool, optional) – check if the pattern matches a short name RINEX. The default is True.
long_name (bool, optional) – check if the pattern matches a long name RINEX. The default is True.
gfz_godc_name (bool, optional) – check if the pattern matches a GFZ’s GODC (GNSS Operational Data Center) internal long name RINEX. The default is True.
compressed (bool or None) – check if the pattern matches a compressed RINEX (True) or not (False) if None, does not matter (return both compressed or not)
specific_sites (list, optional) – Filter only those specific sites. The default is [].
end_epoch (start_epoch and) – Filter the RINEXs between those two epochs (included) Can be for instance start_epoch=dt.datetime(2021,1,1) and end_epoch=dt.datetime(2021,12,31) The default is None.

Returns:

Files_rnx_lis – Found RINEXs list.

Return type:

list

Notes

is very similar with geodetik.rinex_lister, gins_runner.get_rinex_list, operational.multi_finder_rinex

But this one is the most recent and elaborated (July 2022), must be used in priority !!!

geodezyx.operational.download_prods module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_prods.download_gnss_products(archive_dir, startdate, enddate, AC_names=('wum', 'cod'), prod_types=('sp3', 'clk'), remove_patterns=('ULA',), archtype='week', new_name_conv=True, parallel_download=4, archive_center='ign', mgex=False, repro=0, sorted_mode=False, return_also_uncompressed_files=True, ftp_download=False, dow_manu=False)

Download GNSS products from different IGS data centers

Parameters:

archive_dir (str) – the parent directory where the products will be stored.
startdate (datetime) – the start date in regular calendar date.
enddate (datetime) – the end date in regular calendar date..
AC_names (tuple, optional) – the names of the wished analysis centers. It also control the product’s lattency with the new naming convention: simply add it completly in the AC name e.g. IGS0OPSRAP The default is (“wum”,”cod”).
prod_types (tuple, optional) – the wished products. The default is (“sp3”,”clk”).
remove_patterns (tuple, optional) – the patterns you want to exclude. The default is (“ULA”,).
archtype (str, optional) – structure of the local archive sub-directories. see effective_save_dir_orbit function for more details. The default is ‘week’. an alternatiove can be ‘year/doy’.
new_name_conv (bool, optional) – Also handle the new name convention. The default is True.
parallel_download (int, optional) – control parallel download. The default is 4.
archive_center (TYPE, optional) – name of the IGS’s archive/data center. The default is ‘ign’.
mgex (bool, optional) – get MGEX products. The default is False.
repro (int, optional) – get repro products. The default is 0 i.e. operational products.
sorted_mode (bool, optional) – sort the download or not. The default is False.
return_also_uncompressed_files (bool, optional) – in the final list output, return also already downloaded and uncompressedfiles. The default is True.
ftp_download (bool, optional) – DESCRIPTION. The default is False.
dow_manu (int or bool or None, optional) – Control the download for weekly files dow_manu = False, no dow manu, consider the converted dow from the time span, regular case dow_manu = None, no dow in the REGEX, the crawler will search only for the week dow_manu = 0 or 7, the dow in question The default is False.

Returns:

list of the local files’s paths.

Return type:

list

Note

The new naming convention has been fully adopted since GPS Week 2238-0

to control the lattency with the new naming convention, simply add it completly in the AC name e.g. IGS0OPSRAP

geodezyx.operational.download_prods.multi_downloader_orbs_clks(archive_dir, startdate, enddate, calc_center='igs', sp3clk='sp3', archtype='year/doy', parallel_download=4, archive_center='ign', repro=0, sorted_mode=False, force_weekly_file=False, return_also_uncompressed_files=True)

Download IGS products. Can manage MGEX products too (see archive_center argument)

Parameters:

archive_dir (str) – Parent archive directory where files will be stored
enddate (startdate &) – Start and End of the wished period
calc_center (str or list of str) – calc_center can be a string or a list, describing the calc center e.g. ‘igs’,’grg’,’cod’,’jpl’ …
sp3clk (str) –
Product type, can handle :

’clk’

’clk_30s’

’sp3’

’snx’

’sum’

’erp’

’bia’
archive_center (str) –
server of download, “regular” IGS or MGEX, can handle :

’cddis’

’cddis_mgex’

’cddis_mgex_longname’

’ign’

’ign_mgex’

’ign_mgex_longname’

’gfz_local’
archtype (str) –
string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …
repro (int) – number of the IGS reprocessing (0 = routine processing)
sorted_mode (bool) –

if False:
using the map multiprocess fct so the download order will be scrambled

if True:
using the apply multiprocess fct so the download order will be in the chrono. order

The scrambled (False) is better, bc. it doesn’t create zombies processes

Returns:

localfiles_lis – list of downloaded products paths

Return type:

list of str

geodezyx.operational.download_prods.multi_downloader_orbs_clks_2(**kwargs)

geodezyx.operational.download_prods.orbclk_long2short_name(longname_filepath_in, rm_longname_file=False, center_id_last_letter=None, center_manual_short_name=None, force=False, dryrun=False, output_dirname=None)

Rename a long naming new convention IGS product file to the short old convention Naming will be done automaticcaly based on the 3 first charaters of the long AC id e.g. CODE => cod, GRGS => grg, NOAA => noa …

Parameters:

longname_filepath_in (str) – Full path of the long name product file
rm_longname_file (bool) – Remove the original long name product file
center_id_last_letter (str) – replace the last letter of the short AC id by another letter (see note below)
center_manual_short_name (str) – replace completely the long name with this one overrides center_id_last_letter
force (bool) – if False, skip if the file already exsists
dryrun (bool) – if True, don’t rename effectively, just output the new name
output_dirname (str) – directory where the output shortname will be created if None, will be created in the same folder as the input longname

Returns:

shortname_filepath – Path of the short old-named product file

Return type:

str

Note

if you rename MGEX orbits, we advise to set center_id_last_letter=”m” the AC code name will be changed to keep a MGEX convention (but any other caracter can be used too)

e.g. for Bern’s products, the long id is CODE

if center_id_last_letter=None, it will become cod, if center_id_last_letter=m, it will become com

geodezyx.operational.download_rinex module

Created on 23/04/2024 21:31:45

@author: psakic

geodezyx.operational.download_rinex.download_gnss_rinex(statdico, archive_dir, startdate, enddate, archtype='stat', parallel_download=4, user='anonymous', passwd='anonymous@isp.com', path_ftp_crawled_files_save=None, path_ftp_crawled_files_load=None, skip_crawl=False, path_all_ftp_files_save=None, quiet_mode=False, final_archive_for_sup_check=None, force=False, get_rnx2=True, get_rnx3=True)

Parameters:

statdico (dict) –
a statdico is a dictionary associating Archives Centers to list of stations
Exemple:
>>> statdico['archive center 1'] = ['STA1','STA2','STA3', ...] >>> statdico['archive center 2'] = ['STA2','STA1','STA4', ...]
the supported archive center are (april 2024):
igs_cddis or igs (CDDIS data center)

igs_sopac (for the SOPAC/UCSD/SIO data center, but not very reliable)

igs_ign (IGN’s data center, main server at St Mandé)

igs_ign_ensg (IGN’s data center, secondary server at ENSG, Marne-la-Vallée)

* not reimplemented yet * rgp (IGN’s RGP St Mandé center)

rgp_mlv (IGN’s RGP Marne la Vallée center)

rgp_1Hz (IGN’s RGP, all the 24 hourly rinex for the day will be downloaded)

renag

ovsg

unavco

sonel

geoaus (Geosciences Australia)

ens_fr

nav or brdc as archive center allows to download nav files (using ‘BRDC’ as station name) from the ROB server, using GOP files

nav_rt or brdc_rt as archive center allows to download real time nav files from the BKG server * not reimplemented yet *
archive_dir (str) – the root directory on your local drive were to store the RINEXs
archtype (str) –
string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …
passwd (user &) – user & password for a secure server
path_ftp_crawled_files_save (str) – will save at the given path (directory+filename) in a CSV file containing the list of the existing RINEXs found on the server by the FTP crawler. It allows to use this list directly if one face a timeout during the download part.
path_ftp_crawled_files_load (str) – load and use the list of the existing RINEXs found on the FTP server, generated by a previous run of the FTP crawler (called by download_gnss_rinex or directly by ftp_files_crawler). a new call of ftp_files_crawler can be bypassed with skip_crawl.
skip_crawl (bool) – when a list of the existing RINEXs found on the FTP server is provided with path_ftp_crawled_files_load, skip a new call of ftp_files_crawler. Then, we assume the provided list as a complete one, ready for the download step
path_all_ftp_files_save (str) – will save at the given path (directory+filename) in a CSV file ALL the remote files found on the FTP server.
quiet_mode (bool) – List the available RINEXs without downloading them. Useful only if path_ftp_crawled_files_save is given
final_archive_for_sup_check (str) – The final archive path or a file containing the archived RINEXs in their final destination. useful if the final archive is different from archive_dir * not re-implemented yet *
force (bool) – Force the download even if the file already exists locally
get_rnx3 (get_rnx2 &) – limit the search/download to RINEX2 (short names) and/or RINEX3 (long names) depending on the boolean given

Returns:

url_list (list of str) – list of URLs
savedir_list (list of str) – list of downloaded products paths
Minimal exemple
————— – >>> statdic = dict() >>> statdic[‘igs_cddis’] = [‘ZIMM’,’tlse’] >>> archive_dir = ‘/home/USER/test_dl_rnx’ >>> startdate = dt.datetime(2020,1,1) >>> enddate = dt.datetime(2020,1,31) >>> geodezyx.operational.download_gnss_rinex_legacy(statdic, archive_dir, startdate, enddate)

geodezyx.operational.download_rinex.effective_save_dir(parent_archive_dir, stat, date, archtype='stat')

INTERNAL_FUNCTION

archtype =: stat stat/year stat/year/doy year/doy year/stat week/dow OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_rinex.ens_fr(stat, date)

geodezyx.operational.download_rinex.ftp_download_frontend(urllist, savedirlist, parallel_download=1, secure_ftp=False, user='anonymous', passwd='anonymous@isp.com', force=True)

geodezyx.operational.download_rinex.ftp_files_crawler(table, secure_ftp=False, user=None, passwd=None, path_ftp_crawled_files_save=None, path_all_ftp_files_save=None, force=False): filter the table with download_gnss_rinex with an optimized FTP crawl

geodezyx.operational.download_rinex.geoaus_server(stat, date): Geosciences Australia ex : ftp://ftp.ga.gov.au/geodesy-outgoing/gnss/data/daily/2010/10063/

geodezyx.operational.download_rinex.igs_cddis_nav_server(stat, date)

geodezyx.operational.download_rinex.igs_cddis_server(stat, date)

geodezyx.operational.download_rinex.igs_ign_ensg_server(stat, date)

geodezyx.operational.download_rinex.igs_ign_server(stat, date)

geodezyx.operational.download_rinex.igs_sopac_server(stat, date)

geodezyx.operational.download_rinex.nav_bkg_server(stat, date)

geodezyx.operational.download_rinex.nav_rob_server(stat, date)

geodezyx.operational.download_rinex.orpheon_server(stat, date, user='', passwd='')

geodezyx.operational.download_rinex.ovsg_server(stat, date, user='', passwd='')

geodezyx.operational.download_rinex.renag_server(stat, date)

geodezyx.operational.download_rinex.rgp_ign_mlv_server(stat, date)

geodezyx.operational.download_rinex.rgp_ign_smn_1_hz_server(stat, date)

geodezyx.operational.download_rinex.rgp_ign_smn_server(stat, date)

geodezyx.operational.download_rinex.sonel_server(stat, date): ex : ftp://ftp.sonel.org/gps/data/2015/001/

geodezyx.operational.download_rinex.unavco_server(stat, date)

geodezyx.operational.download_rinex.uwiseismic_server(stat, date, user='', passwd='')

geodezyx.operational.download_rinex_legacy module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

geodezyx.operational.download_rinex_legacy.download_gnss_rinex_legacy(statdico, archive_dir, startdate, enddate, archtype='stat', parallel_download=4, sorted_mode=False, user='', passwd='', filter_ftp_crawler=True, path_ftp_crawled_files_save=None, path_ftp_crawled_files_load=None, quiet_mode=False, final_archive_for_sup_check=None, force=False)

Parameters:

statdico (dict) –

a statdico is a dictionary associating Archives Centers to list of stations

Exemple:

>>> statdico['archive center 1'] = ['STA1','STA2','STA3', ...]
>>> statdico['archive center 2'] = ['STA2','STA1','STA4', ...]

the supported archive center are (july 2015):

igs_cddis or igs (cddis center)

igs_sopac (for the sopac/ucsd/sio center, but not very reliable)

rgp (IGN’s RGP St Mandé center)

rgp_mlv (IGN’s RGP Marne la Vallée center)

rgp_1Hz (IGN’s RGP, all the 24 hourly rinex for the day will be downloaded)

renag

ovsg

unavco

sonel

geoaus (Geosciences Australia)

ens_fr

nav or brdc as archive center allows to download nav files (using ‘BRDC’ as station name) from the ROB server, using GOP files

nav_rt or brdc_rt as archive center allows to download real time nav files from the BKG server

archtypestr

string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …

sorted_modebool

if False:: using the map multiprocess fct so the download order will be scrambled
if True:: using the apply multiprocess fct so the download order will be in the chrono. order

The scrambled (False) is better, bc. it doesn’t create zombies processes

user & passwdstr

user & password for a locked server

filter_ftp_crawlerbool

use an improved FTP crawler to find which files actually exist to accelerate the download. If path_ftp_crawled_files_load is given, use this loaded list.

path_ftp_crawled_files_savestr

will save at the given path (directory+filname) in a pickle containing the list of the existing RINEXs found on the server by the FTP crawler. It allows to use this list directly if one face a timeout during the download part. NB for advanced users: the pickle is a tuple (urllist,savedirlist)

path_ftp_crawled_files_loadstr

load and use the list of the existing RINEXs found on the FTP server, generated by a previous run of the FTP crawler (called by download_gnss_rinex or directly by ftp_files_crawler). overrides an internal call of ftp_files_crawler.

quiet_modebool

List the available RINEXs without downloading them. Useful only if path_ftp_crawled_files_save is given

final_archive_for_sup_checkstr

The final archive path or a file containing the archived RINEXs in their final destination. useful if the final archive is different from archive_dir

forcebool

Force the download even if the file already exists locally

Returns:

url_list (list of str) – list of URLs
savedir_list (list of str) – list of downloaded products paths
Minimal exemple
————— – >>> statdic = dict() >>> statdic[‘igs_cddis’] = [‘ZIMM’] >>> archive_dir = ‘/home/USER/test_dl_rnx’ >>> startdate = dt.datetime(2000,1,1) >>> enddate = dt.datetime(2000,1,31) >>> geodezyx.operational.download_gnss_rinex_legacy(statdic, archive_dir, startdate, enddate)

geodezyx.operational.download_rinex_legacy.effective_save_dir(parent_archive_dir, stat, date, archtype='stat')

INTERNAL_FUNCTION

archtype =: stat stat/year stat/year/doy year/doy year/stat week/dow OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_rinex_legacy.ens_fr_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.geoaus_server_legacy(stat, date): Geosciences Australia ex : ftp://ftp.ga.gov.au/geodesy-outgoing/gnss/data/daily/2010/10063/

geodezyx.operational.download_rinex_legacy.igs_cddis_nav_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.igs_cddis_server_legacy(stat, date, user='', passwd='')

geodezyx.operational.download_rinex_legacy.igs_sopac_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.multi_archiver_rinex(rinex_lis, parent_archive_dir, archtype='stat', move=True, force_mv_or_cp=False)

from rinex_lis, a list of rinex (generated by the function multi_finder_rinex)

move (if move=True) of copy (if move=False) those rinexs in the parent_archive_dir according to the archtype, string describing how the archive directory is structured, e.g :

stat

stat/year

stat/year/doy

year/doy

year/stat

week/dow/stat

… etc …

geodezyx.operational.download_rinex_legacy.multi_downloader_rinex(**kwargs)

geodezyx.operational.download_rinex_legacy.nav_bkg_server(stat, date)

geodezyx.operational.download_rinex_legacy.nav_rob_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.orpheon_server_legacy(stat, date, user='', passwd='')

geodezyx.operational.download_rinex_legacy.ovsg_server_legacy(stat, date, user='', passwd='')

geodezyx.operational.download_rinex_legacy.renag_server(stat, date)

geodezyx.operational.download_rinex_legacy.rgp_ign_mlv_server(stat, date)

geodezyx.operational.download_rinex_legacy.rgp_ign_smn_1_hz_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.rgp_ign_smn_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.rnx_long2short_name(longname_filepath_in): MUST BE IMPROVED

geodezyx.operational.download_rinex_legacy.sonel_server_legacy(stat, date): ex : ftp://ftp.sonel.org/gps/data/2015/001/

geodezyx.operational.download_rinex_legacy.unavco_server_legacy(stat, date)

geodezyx.operational.download_rinex_legacy.uwiseismic_server(stat, date, user='', passwd='')

geodezyx.operational.download_utils module

@author: psakic This sub-module of geodezyx.operational contains functions to download gnss data and products from distant IGS servers. it can be imported directly with: from geodezyx import operational The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License Copyright (C) 2019 Pierre Sakic et al. (IPGP, sakic@ipgp.fr) GitHub repository : https://github.com/GeodeZYX/geodezyx-toolbox

class geodezyx.operational.download_utils.MyFTP_TLS(host='', user='', passwd='', acct='', keyfile=None, certfile=None, context=None, timeout=<object object>, source_address=None, *, encoding='utf-8')

Bases: FTP_TLS

Explicit FTPS, with shared TLS session

ntransfercmd(cmd, rest=None)

Initiate a transfer over the data connection.

If the transfer is active, send a port command and the transfer command, and accept the connection. If the server is passive, send a pasv command, connect to it, and start the transfer command. Either way, return the socket for the connection and the expected size of the transfer. The expected size may be None if it could not be determined.

Optional `rest’ argument can be a string that is sent as the argument to a REST command. This is essentially a server marker used to tell the server to skip over any data up to the given marker.

geodezyx.operational.download_utils.downloader(url, savedir, force=False, check_if_file_already_exists_uncompressed=True)

general function to download a file

can also handle non secure FTP

geodezyx.operational.download_utils.downloader_wrap(intup)

geodezyx.operational.download_utils.effective_save_dir_orbit(parent_archive_dir, calc_center, date, archtype='year/doy/')

INTERNAL_FUNCTION

archtype =: stat stat/year stat/year/doy year/doy year/stat week/dow wkwwww : use a GFZ’s CF-ORB wk<wwww> naming OR only ‘/’ for a dirty saving in the parent folder … etc …

geodezyx.operational.download_utils.ftp_downloader(ftp_obj, full_remote_path, localdir): download a file through FTP protocol

geodezyx.operational.download_utils.ftp_downloader_core(ftp_obj, filename, localdir)

do the FTP download, if we are aleady in the right FTP folder

internal function of ftp_downloader

geodezyx.operational.download_utils.ftp_downloader_wo_objects(tupin)

create the necessary FTP object

should not be used anymore

geodezyx.operational.download_utils.ftp_downloader_wrap(intup)

geodezyx.operational.download_utils.ftp_files_crawler_legacy(urllist, savedirlist, secure_ftp): filter urllist,savedirlist generated with download_gnss_rinex with an optimized FTP crawl

geodezyx.operational.download_utils.ftp_objt_create(secure_ftp_inp, host='', chdir='', parallel_download=1, user='anonymous', passwd='')

geodezyx.operational.download_utils.start_end_date_easy(start_year, start_doy, end_year, end_doy)

generates start/end datetimes from a start/end year/day of year

Parameters:

start_year (int) – start year.
start_doy (int) – start day of year.
end_year (int) – end year.
end_doy (int) – end day of year.

Returns:

start (datetime) – converted start datetime.
end (datetime) – converted end datetime.

geodezyx.operational.gins_runner module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the GNSS processing software GINS.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.gins_runner.bad_sat_finder(orbfilein, egrep_ready=True)

geodezyx.operational.gins_runner.check_good_exec_of_GINS(streamin, director_name)

geodezyx.operational.gins_runner.check_if_DOMES_in_oceanloadfile(domes, oclofile)

geodezyx.operational.gins_runner.check_if_file_in_gin_folder(a_file_path, gins_path='')

geodezyx.operational.gins_runner.check_if_stat_in_stationfile(stat, stationfile)

geodezyx.operational.gins_runner.clk_2_gins(clk_pathin)

geodezyx.operational.gins_runner.copy_file_in_gin_folder(a_file_path, repository_folder='/home/psakicki/gin/TP/GWADA/RINEX')

geodezyx.operational.gins_runner.dirs_copy_generik_2_working(director_name_prefix, director_generik_path, temp_data_folder=None)

geodezyx.operational.gins_runner.double90_runner(inp_prarie_file, temp_data_folder='')

geodezyx.operational.gins_runner.double_diff_binom(rinex_path_A, rinex_path_B, temp_data_folder='', final_data_folder=''): DISCONTINUED

geodezyx.operational.gins_runner.double_diff_multi(rinex_path_list, temp_data_folder='', final_data_folder='', remove=True, ignore_glo_gal=False)

rinex_path_list :: List of RINEXs paths
temp_data_folder & final_data_folder :: if empty string (‘’), files are saved in pygins_runner “TEMP_DATA” folder and in the specified folder else

geodezyx.operational.gins_runner.download_convert_2_gins_orb_clk(centraldate, work_folder=None, ac='jpl', repro=2, rm_temp_files=True, data_center='cddis', force=False)

geodezyx.operational.gins_runner.export_results_gins_listing(listings_list_in, outpath, static_or_kinematic='kine', outprefix='', coordtype='FLH')

geodezyx.operational.gins_runner.find_DOMESstat_in_stationfile(stat, stationfile)

geodezyx.operational.gins_runner.gen_dirs_from_double_diff(dd_files_paths_in, director_generik_path, director_name_prefix, out_director_folder='', temp_data_folder='', stations_file='', oceanload_file='', auto_staocl=False, perso_orbclk=False, gins_style_orb='GRG', ac='igs', repro=2, auto_interval=True, out_coords='NULL')

NEW FCT WHICH CAN MANAGE BOTH ONE RINEX OR A LIST OF RINEX

produce a director from a rinex, and write it in the specific location

Input:: rinex_paths_in : can be a rinex path (a string) or a list of paths
Return :: the path of the produced director as a string

If the rinex is not in a gins style folder, it will be copied in the temp_data_folder

If temp_data_folder is not specified ( == ‘’) rinex will be copied in a ad hoc folder ../gin/TEMP_DATA

If out_director_folder is not specified ( == ‘’), output director will be created in the ../gin/data/directeur folder

auto_staoclTrue or False: create automatically a station file and a ocean loading file with the rinex header auto mode is prioritary upon the manu mode so, if a path for stat file or ocload file is specified and auto_staocl is on, it will be the automatic stat/ocload which will be used
perso_orbclkTrue or False: download and use specifics orbits according to ac & repro args (they are useless if perso_orbclk aren’t activated)
auto_intervalTrue or False: find the interval in RINEX and apply it to the director
out_coords‘XYZ’ or ‘FLH’/’PLH’: for geocentrical or geographical coords in output. any other string leaves the type of the generic director.
prairieTrue or False: run prairie externally if True, prairie_kwargs control the arguments of the function prairie_manual (cf above)

geodezyx.operational.gins_runner.gen_dirs_from_rnxs(rinex_paths_in, director_generik_path, director_name_prefix, out_director_folder='', temp_data_folder='', stations_file='', oceanload_file='', auto_staocl=False, perso_orbclk=False, ac='igs', repro=2, auto_interval=True, out_coords='NULL', prairie=False, prairie_kwargs={'with_historik': 1, 'with_wsb': 1})

NEW FCT WHICH CAN MANAGE BOTH ONE RINEX OR A LIST OF RINEX

produce a director from a rinex, and write it in the specific location

Input:: rinex_paths_in : can be a rinex path (a string) or a list of paths
Return :: the path of the produced director as a string

If the rinex is not in a gins style folder, it will be copied in the temp_data_folder

If temp_data_folder is not specified ( == ‘’) rinex will be copied in a ad hoc folder ../gin/TEMP_DATA

If out_director_folder is not specified ( == ‘’), output director will be created in the ../gin/data/directeur folder

auto_staoclTrue or False: create automatically a station file and a ocean loading file with the rinex header auto mode is prioritary upon the manu mode so, if a path for stat file or ocload file is specified and auto_staocl is on, it will be the automatic stat/ocload which will be used
perso_orbclkTrue or False: download and use specifics orbits according to ac & repro args (they are useless if perso_orbclk aren’t activated)
auto_intervalTrue or False: find the interval in RINEX and apply it to the director
out_coords‘XYZ’ or ‘FLH’/’PLH’: for geocentrical or geographical coords in output. any other string leaves the type of the generic director.
prairieTrue or False: run prairie externally if True, prairie_kwargs control the arguments of the function prairie_manual (cf above)

geodezyx.operational.gins_runner.get_director_list(wildcard_dir): with a wildcard (e.g. ‘GWADA_MK2*’) and return a list of corresponding directors found in gin/data/directeur folder

geodezyx.operational.gins_runner.get_gins_path(extended=False)

geodezyx.operational.gins_runner.get_rinex_list(parent_folder, specific_stats=[], invert=False, compressed=True, start=datetime.datetime(1980, 1, 1, 0, 0), end=datetime.datetime(2099, 1, 1, 0, 0))

return :: all RINEXs found in a parent folder and his subfolders (compressed or not)
parent_folder :: can be a the path of the partent folder (the rinex archive path) but also a RINEX list already found (modification of 170524 to gain speed)
specific_stats :: MUST BE a list [‘STA1’,’STA2’,’STA3’] So, if only one elt in specific_stats, use a syntax as : [‘STA1’]
invert :: False = keeping the specific stats True = removing the specific stats or for all stations, leave a empty tuple in specific_stats

NB : the end date is included

geodezyx.operational.gins_runner.get_temp_data_gins_path()

geodezyx.operational.gins_runner.make_path_ginsstyle(pathin): input path must be an absolute path with /gin/ inside output will be .temp.gin/<rest of the path>

geodezyx.operational.gins_runner.merge_yaml(yaml1, yaml2, yaml_out=None)

geodezyx.operational.gins_runner.orbit_cleaner(orbfilein, orbfileout): remove sat of a gins orbit file without a nominal number of epoch

geodezyx.operational.gins_runner.prairie_manual(rinex_paths_in, temp_data_folder='', force=True, with_historik=True, with_wsb=True, argsdict={})

argsdict :: a dictionnary so as argsdict[argument] = val e.g. argsdict[‘-options’] = “/home/psakicki/THESE/SOFTWARES/GINSv2B/gin/data/prairie/options_GPS.dat” argsdict[‘-wsb’] = “/home/psakicki/THESE/SOFTWARES/GINSv2B/gin/data/prairie/WSB_NEW_1996.dat”

argument cc2noncc, wsb, options, and out are automatically managed

geodezyx.operational.gins_runner.run_director_list_wrap(tupinp)

geodezyx.operational.gins_runner.run_director_wrap(intup)

geodezyx.operational.gins_runner.run_directors(dir_paths_in, opts_gins_pc='', opts_gins_90=' ', version='OPERA', fic_mode=False): NEW FCT WHICH CAN MANAGE BOTH ONE RINEX OR A LIST OF RINEX, OR A FIC file (170613)

geodezyx.operational.gins_runner.run_dirs_multislots(director_lis, slots_lis=['', 'U', 'L', 'R'], opts_gins_pc='', opts_gins_90='', version='OPERA', fic_mode=False): run a list of dir in parallel (using different ‘slots’)

geodezyx.operational.gins_runner.smart_directors_to_run(wildcard_dir='', full_path_out=True)

smart runner check directors who worked, and thus give a list of directors whitout those who worked

listing and directeur folders are inspected automatically

geodezyx.operational.gins_runner.smart_listing_archive(wildcard_dir, gins_main_archive, gins_anex_archive, prepars_archive, director_archive): for each listing corresponding to the wildcard : if it’s a prepars => go to the prepars_archive if it’s a gins without duplicate => go to the gins_main_archive if it’s a gins with duplcates => one goes to gins_main_archive

the others in gins_anex_archive

geodezyx.operational.gins_runner.sort_by_stations(archive_path, wildcard, i)

i is the indice of the first character of the station name in eg : i = 18 for filename KARIB_MK3_FLH_v2__bara_22282_2011_003

archive path is ABSOLUTE b.c. it can be outside of the gins folder

geodezyx.operational.gins_runner.sort_orbit_gins(pathin, pathout)

geodezyx.operational.gins_runner.sp3_2_gins(sp3_pathin)

geodezyx.operational.gins_runner.write_oceanload_file(station_file, oceanload_out_file, fes_yyyy=2004)

geodezyx.operational.groops_frontend module

Created on Tue Jan 17 10:54:49 2023

@author: psakicki

geodezyx.operational.groops_frontend.groops_basic_runner(xml_cfg_path='', global_var_dict={}, xml_var_dict={}, dry_run=False, verbosity='ERROR', log_dir=None, groops_bin_path='/opt/softs_gnss/groops/bin/groops')

Parameters:

xml_cfg_path (str, optional) – the XML config file for GROOPS. The default is “”.
global_var_dict (dict, optional) – A dictionnary to change the global variables values, like: global_var_dict[“global_var_name”] = new_value The default is dict().
xml_var_dict (dict, optional) – A dictionnary to change the values in the config XML file. Not implemented yet. The default is dict().
dry_run (bool, optional) – If True print the command but do not run it . The default is False.
verbosity (str, optional) – verbosity level of the console logger use keywords CRITICAL, ERROR, WARNING, INFO, DEBUG The default is ‘ERROR’.
log_dir (str, optional) – If provided, directory where the logs are stored. The default is None.
groops_bin_path (TYPE, optional) – Path of the GROOPS bin. The default is ‘/opt/softs_gnss/groops/bin/groops’.

Return type:

None.

geodezyx.operational.groops_frontend.groops_ppp_full_runner(rinex_path, project_name, igs_ac_10char, cfg_files_dict, log_root_dir, vmf_tropo_root_dir, prods_gnss_root_dir, cfg_files_root_dir, sitelogs_root_dir, groops_bin_path='/opt/softs_gnss/groops/bin/groops', verbosity='ERROR')

High level function to run a GROOPS’s PPP job This function download IGS’s products, convert them, and run the PPP job see the Notes below for more details

Parameters:

rinex_path (str) – the path of the RINEX file to process.
project_name (str) – a personalized name for your processing project
igs_ac_10char (str) – the 10 char. ID for the IGS AC you want to use e.g. ‘COD0OPSFIN’ can handle operational (OPS) and MGEX (MGX) lines
cfg_files_dict (dict) – a dictionary controlling the conversion/processing steps and the corresponding config files.
log_root_dir (str) – directory path where the frontend logs will be written
vmf_tropo_root_dir (str) – directory path where the VMF3/ECMWF grids will be stored (auto download)
prods_gnss_root_dir (str) – directory path where the IGS products (not converted) will be stored (auto download)
cfg_files_root_dir (str) – directory path where the config files are stored.
sitelogs_root_dir (str) – directory path where the sitelogs files are stored.
groops_bin_path (str, optional) – Path of the GROOPS bin. The default is ‘/opt/softs_gnss/groops/bin/groops’.
verbosity (str, optional) – verbosity level of the console logger for the GROOPS’s messages use keywords CRITICAL, ERROR, WARNING, INFO, DEBUG The default is ‘ERROR’. This verbosity level does not apply to this function

Note

the config files must be checked and edited manually to fit your environnement config it is the config files which contains the most useful parameters use groopsGui to help you

prototype for config files are in: …/geodezyx/000_exemples/groops_frontend/configfiles/

do not forget to update on a regular basis GROOPS’s data folder: ` https://ftp.tugraz.at/outgoing/ITSG/groops/data.zip`

Return type:: None.

geodezyx.operational.groops_frontend.log_subprocess(pipe, logger=None, file=None, file2=None): Intern fuction for subprocess_frontend2 to write the stdout/err in the console logger + a logfile

geodezyx.operational.groops_frontend.subprocess_frontend2(cmd_in, save_log=True, log_dir=None, log_name_out='out.log', log_name_err='err.log', logname_timestamp=True, err_also_in_outfile=True, logger_objt_level_out=None, logger_objt_level_err=None)

A generic frontend to run an extern command through subprocess and write the stdout and stderr outputs in log files.

Parameters:

cmd_in (str) – The subprocess command.
save_log (str, optional) – export as log the stdout/stderr in files. The default is True.
log_dir (str, optional) – directory where the logs will be stored. The default is None.
log_name_out (str, optional) – filename of the stdout log. The default is “out.log”.
log_name_err (str, optional) – filename of the stderr log. The default is “err.log”.
logname_timestamp (str, optional) – add a timestamp as prefix. The default is True.
logger_objt_level_out (method of a Logger object, optional) – set the logger level of the stdout messages. can be logger.info or logger.debug for instance. The default is None (logger.info per default then).
logger_objt_level_err (method of a Logger object, optional) – set the logger level of the stderr messages. can be logger.error or logger.critical for instance. The default is None (logger.error per default then).

Returns:

exitcode – the exit code of the command.

Return type:

int

Notes

Inspired by: https://stackoverflow.com/questions/21953835/run-subprocess-and-print-output-to-logging And for the threading: https://stackoverflow.com/questions/6809590/merging-a-python-scripts-subprocess-stdout-and-stderr-while-keeping-them-disti

geodezyx.operational.groops_frontend.vmf_tropo_downloader(output_dir, startdate=datetime.datetime(2019, 1, 1, 0, 0), enddate=datetime.datetime(2019, 1, 1, 0, 0), model='VMF3', version='OP')

geodezyx.operational.hector_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the time series velocities estimation software HECTOR.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.hector_frontend.MJD2dt(mjd_in)

geodezyx.operational.hector_frontend.get_FLH_from_NEUfile(neufilepath)

geodezyx.operational.hector_frontend.keeping_specific_stats(listoffiles, specific_stats, invert=False): if invert = True : NOT keeping BUT removing specific stats

geodezyx.operational.hector_frontend.momfile_trend_processing(inp_momfile, generik_conf_file, outdir='', remove_ctl_file=True)

geodezyx.operational.hector_frontend.multi_momfile_trend_processing(inpdir, generik_conf_file, outdir='', extention='pre.mom', remove_ctl_file=True, specific_stats=(), invert_specific=False)

geodezyx.operational.hector_frontend.multi_neufile_outlier_removing(inpdir, generik_conf_file, outdir='', extention='neu', specific_stats=(), invert_specific=False, remove_ctl_file=True)

geodezyx.operational.hector_frontend.multi_sumfiles_trend_extract(inp_dir, out_dir, out_prefix, raw_neu_dir='', specific_stats=(), invert_specific=False, style='epc'): style = globk OR epc make a GLOBK style .vel file or make a dirty velocity file compatible with EPC

geodezyx.operational.hector_frontend.neufile_outlier_removing(inp_neufile, generik_conf_file, outdir='', remove_ctl_file=True): from a NEU file => removeoutlier preprocessing => 3 MOM files (one per component)

geodezyx.operational.hector_frontend.sumfiles_to_statdico(inpdir, specific_stats=(), invert_specific=False)

this fct search for every sum file in a folder a stat dico contains no data only the paths to the E,N,U sum files statdico[stat] = [path/E.sum,path/N.sum,path/U.sum ]

for each stat getting the 3 ENU sum files

Thoses lists will be send in sumfiles_trend_extract

geodezyx.operational.hector_frontend.sumfiles_trend_extract(listof3ENUsumfile)

geodezyx.operational.hector_frontend.velfile_from_a_list_of_statVsV_tuple(listoftup, out_dir, out_prefix, raw_neu_dir='', style='epc')

geodezyx.operational.midas_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the time series velocities estimation software MIDAS.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.midas_frontend.midas_plot(path_tenu, path_vel='', path_step=''): based on a plot for TimeSeriePoint

geodezyx.operational.midas_frontend.midas_run(tenu_file_path, work_dir='', path_midas_soft='', step_file_path='', with_plot=True, keep_plot_open=True)

geodezyx.operational.midas_frontend.midas_vel_files_2_pandas_DF(vel_files_in)

Convert MIDAS Velocity files to a Pandas DataFrame

Parameters:: vel_files_in (str or list of str) – if list of str, will consider directly the files inside the list if str, can be the path of a single file, or a generic (wilcard) path to consider several files
Returns:: DF
Return type:: Pandas DataFrame

geodezyx.operational.pride_pppar_frontend module

Created on Wed Feb 14 18:01:49 2024

@author: psakic

geodezyx.operational.pride_pppar_frontend.dl_brdc_pride_pppar(prod_parent_dir, date_list)

geodezyx.operational.pride_pppar_frontend.dl_prods_pride_pppar(prod_parent_dir, date_list, prod_ac_name)

geodezyx.operational.pride_pppar_frontend.pride_pppar_mp_wrap(kwargs_inp)

geodezyx.operational.pride_pppar_frontend.pride_pppar_runner(rnx_path_list, cfg_template_path, prod_ac_name, prod_parent_dir, tmp_dir, cfg_dir, run_dir, multi_process=1, cfg_prefix='pride_pppar_cfg_1a', mode='K', options_dic={}, bin_dir=None, force=False, dl_prods=False)

geodezyx.operational.pride_pppar_frontend.pride_pppar_runner_mono(rnx_path, cfg_template_path, prod_ac_name, prod_parent_dir, tmp_dir, cfg_dir, run_dir, cfg_prefix='pride_pppar_cfg_1a', mode='K', options_dic={}, bin_dir=None, force=False, dl_prods=False, default_fallback=False)

geodezyx.operational.pride_pppar_frontend.run_command(command)

geodezyx.operational.rinex_lister_plotter module

@author: psakic

This sub-module of geodezyx.operational generates RINEX timeline plot.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.rinex_lister_plotter.listing_gins_timeline(path, stat_strt, stat_end, date_strt, date_end, suffix_regex='')

find all gins listings in a folder and his subfolders and plot timeline of the avaiable listings

stat_strt,stat_end,date_strt,stat_end : where to find in the name the statname and the date

geodezyx.operational.rinex_lister_plotter.rinex_check_epochs_availability(rinex_path_list)

Gives simple statistics about RINEX avaiability for all the listed stations

Parameters:: rinex_path_list (list) – A list of rinex paths
Returns:: T – a table with statistics.
Return type:: str

geodezyx.operational.rinex_lister_plotter.rinex_lister(path, add_long_names=True)

find all rinex in a folder and his subfolders path can be a string or a tuple of string => manage multi paths :)

is very similar with softs_runner.multi_finder_rinex, gins_runner.get_rinex_list and operational.rinex_finder

Parameters:

path (str) – archive path.
add_long_names (bool, optional) – consider new names. The default is True.

Returns:

rinexfilelist – list of rinex files.

Return type:

list

Notes

operational.rinex_finder must be used in priority !!! (July 2022)

geodezyx.operational.rinex_lister_plotter.rinex_timeline(inputlist_or_paths, start=datetime.datetime(1980, 1, 1, 0, 0), end=datetime.datetime(2099, 1, 1, 0, 0), use_rinex_lister=True, dots_plot=False, jul_date_plot=False, return_figure=False, xlim_start_end=False)

Frontend function to plot the aviable RINEXs

Parameters:

inputlist_or_paths (iterable) – list of rinex file paths or list of archive directories.
start (datetime, optional) – start of the plot time span. The default is dt.datetime(1980,1,1).
end (datetime, optional) – end of the plot time span. The default is dt.datetime(2099,1,1).
use_rinex_lister (bool, optional) – if True, inputlist_or_paths is a list of archive directories. if False, inputlist_or_paths is a list of rinex file paths. The default is True.
dots_plot (bool, optional) – print dots instead of line. This mode should be avoided because it is slower. The default is False.
jul_date_plot (bool, optional) – Plot the date in julian dates. The default is False.
return_figure (bool, optional) – returns figure. The default is False.
xlim_start_end (bool, optional) – Force start and end values to be the x axis limit The default is False.

Returns:

datadico (dict) – datadico.
fig (matplotlib figure) – figure

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico(inputlist_or_paths, use_rinex_lister=True, optional_info='')

Generate a RINEX datadico

Parameters:

inputlist_or_paths (iterable) – list of rinex file paths or list of archive directories.
use_rinex_lister (bool, optional) – if True, inputlist_or_paths is a list of archive directories. if False, inputlist_or_paths is a list of rinex file paths. The default is True.
optional_info (str, optional) – A addtional information for the rinexs found. Usually it is the archive name The default is ‘’.

Returns:

datadico – datadico[stat] = [(rinexname1,optional1,date1) … (rinexnameN,optionalN,dateN)]

Return type:

dict

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico_merge(datadico_list, priority_list=None)

Merge different RINEXs datadico, produced by rinex_timeline_datadico coming from different archives

Parameters:

datadico_list (list of dict) – list of RINEX datadico.
priority_list (list, optional) – priority list of ‘optional_info’ (archive ID) it will erase optional_info of lower priority NB : it is not very useful, just sort datadico_list in the right order …. The default is None.

Returns:

datadico_out – a merged datadico

Return type:

dict

geodezyx.operational.rinex_lister_plotter.rinex_timeline_datadico_merge_not_very_smart(datadico_list, priority_list)

Merge different RINEXs datadico, produced by rinex_timeline_datadico coming from different archives Args :

rinex_timeline_datadico : list of RINEX datadico priority_list : priority list of ‘optional_info’ (archive ID)

it will erase optional_info of lower priority

Returns :: datadico_out : a merged datadico

geodezyx.operational.rinex_lister_plotter.timeline_plotter(datadico, start=datetime.datetime(1980, 1, 1, 0, 0), end=datetime.datetime(2099, 1, 1, 0, 0), dots_plot=False, jul_date_plot=False, datadico_anex_list=[], use_only_stats_of_main_datadico=False, colordico_for_main_datadico=None, xlim_start_end=False)

Parameters:

datadico (dict) – a RINEX datadico (see rinex_timeline_datadico for details).
start (datetime, optional) – start of the plot time span. The default is dt.datetime(1980,1,1).
end (datetime, optional) – end of the plot time span. The default is dt.datetime(2099,1,1).
dots_plot (TYPE, optional) – print dots instead of line. This mode should be avoided because it is slower. The default is False.
jul_date_plot (bool, optional) – Plot the date in julian dates. The default is False.
datadico_anex_list (list, optional) – A list of secondary datadicos. The default is [].
use_only_stats_of_main_datadico (bool, optional) – Use only stats of main datadico. Advanced usage. The default is False.
colordico_for_main_datadico (dict, optional) – Color for the main datadico. Advanced usage. The default is None.
xlim_start_end (bool, optional) – Force start and end values to be the x axis limit The default is False.

Returns:

fig – figure

Return type:

matplotlib figure

Notes

A simpler version has been commited to geodezyx toolbox GitHub for archive on 20180118 15:59A

geodezyx.operational.rinex_utils module

@author: psakic

This sub-module of geodezyx.operational contains functions to manipulate RINEX files (version 2, quite obsolate).

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.rinex_utils.check_if_compressed_rinex(rinex_path)

geodezyx.operational.rinex_utils.crz2rnx(rinex_path, outdir='', force=True, path_of_crz2rnx='CRZ2RNX'): assuming that CRZ2RNX is in the system PATH per default

geodezyx.operational.rinex_utils.crz2rnx_bad(crinex_in_path, outdir='', force=True, path_of_crz2rnx='CRZ2RNX'): assuming that CRZ2RNX is in the system PATH per default

geodezyx.operational.rinex_utils.read_rnx_epoch_line(line, rnx2=True)

read the epoch line of a RINEX (2 or 3) and extract it as a datetime

Parameters:

line (string) – input line, assuming it’s a RINEX epoch line.
rnx2 (TYPE, optional) – True if RINEX2, False if RINEX3/4. The default is True.

Returns:

epochdt – The epoch of the line.

Return type:

datetime

geodezyx.operational.rinex_utils.rinex_read_epoch(input_rinex_path_or_string, interval_out=False, add_tzinfo=False, out_array=True, out_index=False)

Read the epochs contained in a RINEX File. Can handle RINEX 2 and 3

Parameters:

input_rinex_path_or_string (see below) – input RINEX. can be the path of a RINEX file as string or as Path object, or directly the RINEX content as a string, bytes, StringIO object or a list of lines
interval_out (bool, optional) – output also the intervals. The default is False.
add_tzinfo (bool, optional) – add timezone information in the datetime’s Epoches. The default is False.
out_array (bool, optional) – output results as array. The default is True.
out_index (bool, optional) – output also the index of the epoch line. The default is False.

Returns:

the epochs in the RINEX.

Return type:

array or list

geodezyx.operational.rinex_utils.rinex_renamer(input_rinex_path, output_directory, stat_out_name='', remove=False)

geodezyx.operational.rinex_utils.rinex_sats_checker(p_rnx)

Check the consistency of a RINEX’s sat list for each epoch

Designed for checking the consistency of IPGP-OVPF corrupted RINEXs RINEX-2 only

Parameters:: p_rnx (str) – input RINEX path.
Return type:: bad RINEX path or None.

geodezyx.operational.rinex_utils.rinex_session_id(first_epoch, last_epoch, full_mode=False)

full_mode:: gives the letter of the starting session & the length in hour of the session

geodezyx.operational.rinex_utils.rinex_spliter(input_rinex_path, output_directory, stat_out_name='', interval_size=24, compress=False, shift=0, inclusive=False, teqc_cmd='teqc')

if shift != 0: the start/end of a session is shifted of shift minutes

inclusive: delta of exaclty interval_size => add the 1st epoch of the next sess not inclusive: delta of interval_size - 1s

geodezyx.operational.rinex_utils.rinex_spliter_gfzrnx(input_rinex_path, output_directory, stat_out_name='', interval_size=86400, shift=0, inclusive=False, gfzrnx_cmd='GFZRNX', output_name='::RX3::', custom_cmds='')

geodezyx.operational.rinex_utils.rinex_start_end(input_rinex_path, interval_out=False, add_tzinfo=False, verbose=True, safety_mode=True)

Return the first and the last epoch of a RINEX file (based on the actual content of the file, not the header)

Can handle RINEX 2 and 3

Parameters:

input_rinex_path (TYPE) – path of the rinex file. can be the path of a RINEX or directly the RINEX content as a string
interval_out (bool, optional) – output also the intervals. The default is False.
add_tzinfo (bool, optional) – add timezone information in the datetime’s Epoches. The default is False.
verbose (bool, optional) – verbose mode. The default is True.
safety_mode (TYPE, optional) – if the epoch reading fails (e.g. in case of a compressed RINEX) activate a reading of the header and the file name as backup. The default is True.

Returns:

First, las epoches and interval if asked.

Return type:

first_epoch , last_epoch , [interval]

geodezyx.operational.rinex_utils.rinexs_table_from_list(rnxs_inp, site9_col=False, round_date=False, path_col=True)

From a simple RINEX list, summarize the data in an ad-hoc DataFrame

Parameters:: rnxs_inp (iterable or str) – if iterable, a list of RINEX files if str, path of an input RINEX list as text file.
Returns:: DF – A DataFrame with the RINEX info in it.
Return type:: Pandas DataFrame

geodezyx.operational.rinex_utils.rnx2crz(rinex_path, outdir='', force=True, path_of_rnx2crz='RNX2CRZ'): assuming that RNX2CRZ is in the system PATH per default

geodezyx.operational.rinex_utils.same_day_rinex_check(rinex1, rinex2)

geodezyx.operational.rinex_utils.teqc_qc(rinex_path, quick_mode=False, optional_args=''): quick mode : reduced qc and no summary file written

geodezyx.operational.rtklib_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the GNSS processing software RTKLIB.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.rtklib_frontend.read_conf_file(filein)

geodezyx.operational.rtklib_frontend.rtklib_run_from_rinex(rnx_rover, rnx_base, generik_conf, working_dir, experience_prefix='', rover_auto_conf=False, base_auto_conf=True, XYZbase=[0, 0, 0], outtype='auto', calc_center='igs')

auto_conf :

read the header of the rinex and write the conf. file according to it if the mode is disabled, the antenna/rec part of the conf. file will be the same as the generic one

NB : RTKLIB “core” have it’s own reading header option. Thus, my advice is to disable the auto mode for the rover and leave ant1-postype=rinexhead in the generic conf file and enable it for the base with a XYZbase vector initialized or the good XYZ in header on the rinex (the XYZbase vector is prioritary over the base RINEX header)

(but in anycase, prepars good rinex’s headers ;) )

outtype :

‘auto’ (as defined in the generic config file) or ‘dms’ ‘deg’ ‘xyz’ ‘enu’ can manage the upper case XYZ or FLH

geodezyx.operational.track_frontend module

@author: psakic

This sub-module of geodezyx.operational contains functions to run the GNSS processing software TRACK.

it can be imported directly with: from geodezyx import operational

The GeodeZYX Toolbox is a software for simple but useful functions for Geodesy and Geophysics under the GNU LGPL v3 License

geodezyx.operational.track_frontend.run_track(temp_dir, exp_full_name, out_conf_fil, date, rnx_rover)

geodezyx.operational.track_frontend.track_runner(rnx_rover, rnx_base, working_dir, experience_prefix, XYZbase=[], XYZrover=[], outtype='XYZ', mode='short', interval=None, antmodfile='~/gg/tables/antmod.dat', calc_center='igs', forced_sp3_path='', const='G', silent=False, rinex_full_path=False, run_on_gfz_cluster=False, forced_iono_path='')