Skip to content

read SPOTGINS ENU v2 & v3 #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from
Open

read SPOTGINS ENU v2 & v3 #183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
38c776f
add both spotgins v2 & v3
Sep 22, 2025
99eead2
add in conf file
Sep 22, 2025
2e38f40
add header spotgins 2&3
Sep 22, 2025
a836297
indentation
Sep 22, 2025
4772fb3
update comment for clarity in readfmtdata_gnss function
Sep 22, 2025
aabfb30
correct usgs-rneu
Sep 22, 2025
aa07190
add support for 'spotgins-enu-v2' format in readfmtdata_gnss function
Sep 22, 2025
4dcc189
add 'spotgins-enu' format support in readfmtdata function
Sep 22, 2025
8430054
refactor comments for clarity and consistency in spotgins format hand…
Sep 22, 2025
cae3b79
refactor spotgins format handling: update descriptions for clarity an…
Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CODE/etc/rawformats.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ winston|EARTHWORM|EarthWorm Winston Wave Server data request||host:port
gipsy|GNSS|JPL GIPSY-OASIS .tdp file|ANTENNA,RECEIVER,XYZ|fullpath of root directory containing YYYY/FID/*.tdp files
gipsyx|GNSS|JPL GipsyX .tdp file|ANTENNA,RECEIVER,XYZ|fullpath of root directory containing YYYY/FID/*.tdp files
globkval|GNSS|MIT GAMIT/GLOBK VAL file||fullpath of directory containing the .VAL file
spotgins-enu-v2|GNSS|SPOTGINS solutions - version 2 (<2025-08)||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-ippp|GNSS|SPOTGINS IPPP time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-enu|GNSS|SPOTGINS ENU solutions - v2&3||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
spotgins-ippp|GNSS|SPOTGINS IPPP time series (ITES)||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
gamit-pos|GNSS|MIT GAMIT/GLOBL POS time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
pbogps-pos|GNSS|PBO GPS POS time series||fullpath of file(s) with bash wildcard facilities, possible $FID, or URL to single file
usgs-rneu|GNSS|USGS RNEU text file||fullpath of file(s) with bash wildcard facilities
Expand Down
2 changes: 1 addition & 1 deletion CODE/matlab/readfmtdata.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
case {'miniseed','seedlink','arclink','combined','fdsnws-dataselect'}
D(n) = readfmtdata_miniseed(WO,P,N(n),F);

case {'globkval','gipsy','gipsyx','gipsy-tdp','usgs-rneu','ies-neu','ogc-neu','ingv-gps','sbe37-ascii','spotgins-enu-v2','spotgins-ippp','gamit-pos','pbogps-pos'}
case {'globkval','gipsy','gipsyx','gipsy-tdp','usgs-rneu','ies-neu','ogc-neu','ingv-gps','sbe37-ascii','spotgins-enu','spotgins-ippp','gamit-pos','pbogps-pos'}
D(n) = readfmtdata_gnss(WO,P,N(n),F);

case {'hyp71sum2k','fdsnws-event','scevtlog-xml'}
Expand Down
97 changes: 81 additions & 16 deletions CODE/matlab/readfmtdata_gnss.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@
% data format: jjjjj.jj E N V dE dN dV yyyymmddhhmmss yyyy.yyyyyyyyy
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)
%
% format 'spotgins-enu'
% type: SPOTGINS solutions - supports v2 and v3 formats (auto-detection)
% filename/url: P.RAWDATA (use $FID to point the right file/url)
% data format v2: jjjjj.jj E N V dE dN dV yyyymmddhhmmss yyyy.yyyyyyyyy
% data format v3: MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp CorrEN CorrEU CorrNU yyyy-mm-ddTHH:MM:SS DecimalYear Const Flag DateOfExe GinsVersion PrairieVersion
% node calibration: no .CLB file or 4 components (East, North, Up) in meters and (Orbit)READFMTDATA_GNSS subfunction of readfmtdata.m
%
% format 'spotgins-ippp'
% type: GINS IPPP solutions
% filename/url: P.RAWDATA (use $FID to point the right file/url)
Expand Down Expand Up @@ -306,19 +313,66 @@
end
%D.ITRF_YEAR = 'ITRF08';


% -----------------------------------------------------------------------------
case 'spotgins-enu-v2'
% format exemple
%#jjjjj.jjjjjjjj _____E _____N _____U ____dE ____dN ____dU yyyymmddHHMMSS yyyy.yyyyyyy Const Dateofexe GinsVersion
% 52670.83876160 0.055822 0.051638 0.005578 0.001263 0.001163 0.004899 20030131200749 2003.0844898 G 250404_185253 VALIDE_24_2
% 52671.50195600 0.057207 0.054240 -0.004722 0.000705 0.000619 0.002600 20030201120249 2003.0863067 G 250404_185253 VALIDE_24_2

case 'spotgins-enu'
% Supports both v2 and v3 SPOTGINS formats
%
% v2 format example:
%#jjjjj.jjjjjjjj _____E _____N _____U ____dE ____dN ____dU yyyymmddHHMMSS yyyy.yyyyyyy Const Dateofexe GinsVersion
% 52670.83876160 0.055822 0.051638 0.005578 0.001263 0.001163 0.004899 20030131200749 2003.0844898 G 250404_185253 VALIDE_24_2
% 52671.50195600 0.057207 0.054240 -0.004722 0.000705 0.000619 0.002600 20030201120249 2003.0863067 G 250404_185253 VALIDE_24_2
%
% v3 format example:
%#MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp CorrEN CorrEU CorrNU yyyy-mm-ddTHH:MM:SS DecimalYear Const Flag DateOfExe GinsVersion PrairieVersion
% 51668.5 0.080276 -2.058358 0.009854 0.000312 0.000439 0.001339 -0.028963 -0.085824 -0.011884 2000-05-04T12:00:00 2000.340164 G 0 250910_120313 25_1 v56
% 51669.5 0.077154 -2.057207 0.005923 0.000257 0.000371 0.001109 -0.017412 -0.060937 -0.124317 2000-05-05T12:00:00 2000.342896 G 0 250910_120313 25_1 v56

fdat = sprintf('%s/%s.dat',F.ptmp,N.ID);
wosystem(sprintf('rm -f %s',fdat),P);

% Detect format version from first file
format_version = '';
if ~isempty(F.raw)
first_raw = F.raw{1};
if strncmpi('http',first_raw,4)
[s,header] = wosystem(sprintf('curl -s -S "%s" | head -20',first_raw),P);
else
[s,header] = wosystem(sprintf('head -20 %s',first_raw),P);
end
if s == 0
if ~isempty(strfind(header,'SPOTGINS SOLUTION [POSITION] v3')) || ~isempty(strfind(header,'MJD DispEast'))
format_version = 'v3';
elseif ~isempty(strfind(header,'SPOTGINS SOLUTION [POSITION] v2')) || ~isempty(strfind(header,'jjjjj.jjjjjjjj'))
format_version = 'v2';
else
% Try to detect from data columns (fallback method)
if strncmpi('http',first_raw,4)
[s,sample] = wosystem(sprintf('curl -s -S "%s" | grep -v "^#" | head -1',first_raw),P);
else
[s,sample] = wosystem(sprintf('grep -v "^#" %s | head -1',first_raw),P);
end
if s == 0 && ~isempty(sample)
cols = length(strsplit(strtrim(sample)));
if cols >= 17 % v3 has 17+ columns
format_version = 'v3';
elseif cols >= 11 % v2 has 11+ columns
format_version = 'v2';
end
end
end
end
end

if isempty(format_version)
format_version = 'v2'; % default fallback
fprintf('%s: ** INFO ** Could not detect SPOTGINS format version, assuming v2.\n',wofun);
else
fprintf('%s: ** INFO ** Detected SPOTGINS format %s.\n',wofun,format_version);
end

for a = 1:length(F.raw)
fraw = F.raw{a};
cmd0 = sprintf('awk ''/^[^#]/ {print}'' >> %s',fdat); % removes header lines
cmd0 = sprintf('awk ''/^[^#]/ {print}'' >> %s',fdat); % removes header lines
if strncmpi('http',fraw,4)
s = wosystem(sprintf('curl -s -S "%s" | %s',fraw,cmd0),P);
if s ~= 0
Expand All @@ -331,20 +385,31 @@
fprintf('%s: ** WARNING ** Raw data "%s" not found.\n',wofun,fraw);
end
end


% load the file

% load the file
if exist(fdat,'file')
dd = dlmread(fdat);
else
dd = [];
end

if ~isempty(dd)
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % North(mm),East(mm),Up(mm) => E(m),N(m),U(m),Orbit
e = dd(:,5:7);
switch format_version
case 'v3'
% v3 format: MJD DispEast DispNorth DispUp SigmaEast SigmaNorth SigmaUp ...
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % DispEast,DispNorth,DispUp,Orbit => E(m),N(m),U(m),Orbit
e = dd(:,5:7); % SigmaEast,SigmaNorth,SigmaUp

case 'v2'
% v2 format: jjjjj.jj E N U dE dN dU ...
t = dd(:,1) + 678941.5007; % converts MJD to datenum
d = [dd(:,2:4),zeros(size(dd,1),1)]; % E,N,U,Orbit => E(m),N(m),U(m),Orbit
e = dd(:,5:7); % dE,dN,dU
end

e(e<min_error) = min_error;
fprintf('%d data imported.\n',size(dd,1));
fprintf('%d data imported (format %s).\n',size(dd,1),format_version);
else
fprintf('no data found!\n')
t = [];
Expand All @@ -355,7 +420,7 @@

% -----------------------------------------------------------------------------
case 'spotgins-ippp'
% From J.S. - ITES Strasbourg
% From J.S. - ITES Strasbourg
% format example
% !yyyymmdd hhmmss yyyy.yyyyyyyyy jjjjj.jj X_position Y_position Z_position dX dY dZ E N V dE dN dV
% 20160723 65619 2016.558521561 57592.29 4182067.152057 570976.439258 4765940.539811 0.000611 0.000218 0.000673 -0.006574 -0.008848 -0.014844 0.000205 0.000307 0.000859
Expand Down