[FROZEN] Re-add ivo_epoch_prop

[msdemlei]

This is an attempt to resolve bug #19.

(also, a minor editorial addition to the sexagesimal translation functions)

[mbtaylor]

Why does a NULL input parallax require a NULL output rv? If the rv is OK on input but you don't happen to have a parallax for it, the same rv should be a reasonable value for the propagated version. Am I missing something?

[msdemlei]

On Wed, May 15, 2024 at 07:28:08AM -0700, Mark Taylor wrote: @mbtaylor commented on this pull request. Why does a NULL input parallax require a NULL output rv? If the rv is OK on input but you don't happen to have a parallax for it, the same rv should be a reasonable value for the propagated version. Am I missing something?

The trouble is that the common algorithm for epoch propagation ("Lindegren") reconstructs the actual space motion. Or tries to. With a parallax of 0, this would result in infinite velocities, and thus in that case some "very far" is assumed. That still yields huge space velocities and hence, when transformed back to spherical, huge radial velocities. What one could do is restore the non-transformed RV. Is that better than nulling it out? Perhaps, because people still have the RV in the output vector. On the other hand, perhaps not, because we'd be claiming it somehow was a transformed RV when it isn't, and of course people have the original RV anyway. I also like the symmetry: If a component of PM is missing, then the whole transformed PM is NULL. If a component of the radial axis is missing, then the whole radial axis is NULL. But I give you I'm cheating here, too, because we do keep the parallax when the RV is missing. I could be swayed to null out the parallax in this case, too, I think, because arguably RVs *could* change the parallax appreciatably. But then my gut feeling (no computation done) is that there are basically no objects out there where you'd notice that within 100 years or so.

[msdemlei]

ivo_epoch_prop is gone from master again, as the ESAVO implementation differs from what is described here. This PR should remain open until a consensus about the signature is reached (or closed if we decide ivo_epoch_prop is not worth it at all).

[mbtaylor]

I prefer to restore the non-transformed RV in case of NULL parallax.

The use case I'm thinking of, and my guess is it's the dominant one for users of this UDF, is that they want the best available estimate of the astrometric quantities for an epoch in the future/past, given available information. If available information is good enough to improve the values at the current epoch, then do it. If not, fall back to the current epoch values. And don't fabricate a definite value for a quantity at the new epoch if it didn't have one at the current epoch.

So applied to position that means: if there's a PM, apply that to get a new position, and if not, use the current position. Applied to RV it means: if there's a parallax, apply it to get a new RV, and if not, use the current RV.

I think(?) that your proposal to NULL output RV in case of NULL input parallax would be treating the RV/parallax case differently than the position/PM case. For consistency with your proposal you'd want to NULL output position if input PM is missing. That would be reasonable behaviour, but as I commented in #19 I don't think it's what most people are after here.

[msdemlei]

On Thu, May 16, 2024 at 05:02:03AM -0700, Mark Taylor wrote: I prefer to restore the non-transformed RV in case of NULL parallax.

Well, since nobody argues against it and I have no strong leanings either way, I have changed the document accordingly (or so I think), and I have changed the implementation in the pgsphere PR accordingly.

[mbtaylor]

Thanks. Checking my code against the examples here threw up some inconsistencies with my NULL processing as well, so thanks for providing all those too.

[jontxu]

I see Mark has approved the changes... are we waiting for a PR in pgsphere before this one can be merged?

[msdemlei]

On Wed, May 22, 2024 at 08:33:33AM -0700, Jon J. wrote: I see Mark has approved the changes... are we waiting for a PR in pgsphere before this one can be merged?

No, we're waiting for ESAC. So, the epoch_prop removal should be merged now (for the next EN, which should go ahead), and this PR should stay frozen until ESAC or someone else takes it up. Meanwhile, it's true that for the nulling behaviour in DaCHS to match what is specified here, I'd have to roll out a bleeding-edge pgsphere on the production site. But that's a side show compared to the differing signatures on ESAC and g-vo.

[mbtaylor]

I have brought this to ESAC's attention (JIRA issue C9GACS-825, only visible to DPAC members).

[mbtaylor]

I came across two very minor issues with the examples in the text while using it to write unit tests for my code:

1. the use of NaN and NULL is not quite consistent: blank values in returned arrays are mostly represented as NULL but there is a NaN parallax in the return value for the third example. I don't really care whether output blank values are written as NULL or NaN but it should be consistent.
2. The unchanged ra value is propagated from 7.606083572 to 7.606083572000001 in the fourth example. Given precision expectations this isn't wrong, but it's slightly more confusing than it could be, so clarity might be improved by chopping off that trailing 000001.

[msdemlei]

On Fri, May 24, 2024 at 05:17:25AM -0700, Mark Taylor wrote: 1. the use of NaN and NULL is not quite consistent: blank values in 2. The unchanged `ra` value is propagated from 7.606083572 to 7.606083572000001 in the fourth example. Given precision

Yes on both, thanks. It's in commit df9e3eb.

[mbtaylor]

This example is a bit weird - pmra defined but pmdec NULL on input. It's not a very likely case, and the most sensible output is debatable - you could argue that instead of nulling out both PM elements on output it should leave the known pmra intact and return exactly the same output as input. Was it actually intended like this? I wonder if it's supposed to be

ivo_epoch_prop(7.606083572, 11.79044105, 125,  NULL, NULL, 21, 2016.0, 1992.25)

instead - which I think would be a more instructive example.

[msdemlei]

On Wed, Jun 05, 2024 at 04:07:12AM -0700, Mark Taylor wrote: > +\example \begin{lstlisting} +ivo_epoch_prop(7.606083572, 11.79044105, 125, + 21, NULL, NULL, 2016.0, 1992.25) +\end{lstlisting} +\becomes [7.606083572, 11.79044105, 125.0, NULL, NULL, NULL] This example is a bit weird - `pmra` defined but `pmdec` NULL on input. It's not a very likely case, and the most sensible output

It's supposed to illustrate this corner case, so it's weird by design.

is debatable - you could argue that instead of nulling out both PM elements on output it should leave the known `pmra` intact and return exactly the same output as input. Was it actually intended like this? I wonder if it's supposed to be

The "when either PM is NULL, no PMs are applied and no PMs are output" is actually supposed to be like this. We can repeat the parallax/RV debate here, but this time I'm a lot more confident that discarding anything where there's a PM component missing is the right thing to do. You're absolutely right that it's a corner case that will be very rare in practice, but I wanted to still specify what implementors should do.

[mbtaylor]

Fair enough, thanks for the clarification.

[gmantele]

I am no expert here and I completely rely on both of you for this issue.
Is it still supposed to be frozen or could it be merged?

Do you have any news from ESAC guys? If not, should we ping them again?

[mbtaylor]

I believe that @msdemlei marked this PR frozen since we only have one implementation of the 8-parameter ivo_epoch_prop_pos UDF under discussion, namely Markus's.

ESAC have said (issue C9GACS-825, restricted access):.

• We (at ESDC) will create a new ivo_epoch_prop UDF which will differ from esdc_epoch_prop:
• It will take radial velocity as input and will return radial velocity as output (so the argument will become symmetric);
• It will adhere to the new agreements reached for the NULL behaviour;
• I will update this ticket accordingly and will then let it enter a future sprint. Since we are focused on a massive GACS infrastructure migration for the coming half year, this may shift to 2025.

So they do intend to provide an equivalent implementation in GACS, but it may not be soon.
Options for progress:

• Wait until ESDC implement it before merging
• Take ESDC's agreement in principle as sufficient for the multiple implementations criterion and merge
• Persuade somebody else (@jontxu?) to implement it and then merge

[jontxu]

Persuade somebody else (@jontxu?) to implement it and then merge

Could be done but... would it be considered a second implementation if it uses the same codebase? As it would also be based on pgsphere.

[msdemlei]

On Wed, Jul 03, 2024 at 03:57:08AM -0700, Jon J. wrote: >Persuade somebody else ***@***.***?) to implement it and then merge Could be done but... would it be considered a second implementation if it uses the same codebase? As it would also be based on pgsphere.

That wouldn't be my worst concern; the important thing is that someone else has thought about the interfaces and proposals. But in this case I'm really all for waiting for ESAC; it's their function, and they'll notice when something is wrong. And: it's rather esoteric to begin with, so there's no particular hurry either.

[gmantele]

Thank you to all of you for your answers. We will wait for ESAC then 🙂