Re: Raw & Field Data final

Well, we had one "yes" (thanks, Bill) and no "no"s or "yes, but.."s, so I guess it's safe to assume that these two definitions are near enough for now. I'll update the website and advise when it's done.

A separate email will start on the next terms.

Peter

At 16:00 13-01-03 +1100, Peter MATTHEWS wrote:
>There have been no further comments on these two definitions, so it's time
>to settle them and move on. They are dated 20-Dec-02 and are at the bottom
>of this posting. The square brackets themselves, which show the bits most
>recently changed, will be removed of course.
>
>Could you therefore please make a short posting saying whether you can
>accept these definitions or not.
>
>They may not be perfect, but hopefully they are now acceptable enough, and
>if in later discussions we find we need to modify them further, then this
>can certainly be done.
>
>If they are generally accepted, then I'll add them to the web site and we
>can move on to the next set of terms.
>
>Peter
>
>At 05:37 02-01-03 +1100, Peter MATTHEWS wrote:
>>This was originally sent on 20 Dec but the CaveXML server was having
>>problems, so here it is again. - Peter
>>
>>At 20:16 17-12-02 -0500, Lev Bishop wrote:
>>>On Sat, 7 Dec 2002, Peter MATTHEWS wrote:
>>>
>>> > Raw data:
>>> > An initial unaltered digital copy of some or all of the survey Field
>>> Data,
>>> > now ready for processing.
>>>
>>>I'm not entirely sure about "now ready for processing". If the original
>>>data had errors in it that make it impossible to process in its original
>>>form then I think still you want to be able to enter it as raw data. It
>>>should only be at the edited data stage (if at all...) where requirements
>>>like "angles have to be between 0 and 360 degrees", "stations have to have
>>>unique identifiers", needing to have values for all of the instruments for
>>>each leg (eg. if you forgot to record the horizontal angle on a
>>>near-vertical leg and will need to fudge it). On the other hand the format
>>>of the raw data should either be the same as the edited data so that if it
>>>meets all the requirements it can then be processed, or it should be such
>>>that if those requirements are satisfied then it can be automatically
>>>processed into edited data. Maybe "now ready for processing" should be
>>>changed to "in a format suitable for processing, but with no/minimal data
>>>verification or consistancy checking"?
>>
>>
>>Lev has a good point - "now ready for processing" is fairly ambiguous,
>>which is what we are trying to avoid. In my mind the "processing"
>>included manual processing, such as editing mistakes out of it, but on
>>the other hand it could also be interpreted as 'now ready for
>>calculation', which of course may not be the case, as Lev has pointed out.
>>
>>
>>>...
>>> > 4. If such a program unilaterally alters the data as it is being entered,
>>> > then the data has become Edited Data because its values differ from the
>>> > Field Data, i.e. a Raw Data version has effectively been skipped.
>>>Add: "This applies even if a number entered as 1.0 becomes 1.00, where the
>>>numerical value of the data hasn't changed but it is still different from
>>>what was entered". Controversial? I'm thinking in terms of a number
>>>recorded as 125 degrees - or was it 12.5 degrees? - maybe the book person
>>>wasn't careful to make decimal points obvious. Sketch and the loop
>>>closures suggest 12.5 would be more consistent so we go back to the raw
>>>data and look and see whether it was recorded as 125 or 125.0 to decide on
>>>this (assuming the field data isn't available any more for whatever
>>>reason).
>>
>>
>>Another good point, however there's so many possible cases here that I
>>don't think we afford to go into detail. But we could beef up the comment
>>in general terms, to make it clearer.
>>
>>
>>>Other than the above comments, sounds good to me,
>>>
>>>Lev
>>
>>So the new proposed definitions become:
>>Temporary [] brackets identify the bits which I have changed.
>>
>>============================= 20-Dec-02 =================================
>>Field data:
>>The unaltered survey data (readings and/or sketches) recorded in the
>>field by whatever means, or verified copies thereof.
>>Comments:
>>For example, paper-based records, or decipherable images thereof which
>>may have been altered but only to clarify the original data [(for
>>example, marked up photocopies)], or data downloaded unaltered from an
>>instrument (survey instrument, PDA, laptop, etc), or data still stored
>>and observable in an instrument.
>>========================================================================
>>
>>============================= 20-Dec-02
>>======================================
>>Raw data:
>>An initial unaltered digital copy of some or all of the survey Field
>>Data, now ready for [editing, validity checking, calculation or other
>>processing].
>>Comments:
>>1. Where the Field Data was already in digital form, e.g. downloaded from
>>an instrument, the Raw Data version could be an identical but editable
>>copy, whereas the Field Data version would effectively be a read-only copy.
>>2. Raw Data could include sketches now converted to fixed or editable
>>digital form.
>>3. The Raw Data could be in any format, including that of a survey
>>processing program into which the Field Data has been typed.
>>4. If such a program unilaterally alters the data [in any material way]
>>as it is being entered, then the data has become Edited Data because [it
>>differs] from the Field Data, i.e. a Raw Data version has effectively
>>been skipped.
>>5. Such a survey program might also store the data in a proprietary
>>binary format unconducive to easy data exchange, therefore the coming
>>CaveXML standard may need to define suitable forms of Raw Data to allow
>>free exchange. Such binary data after being exported to a text format
>>might qualify. Acceptable formats for raster or vector graphical data may
>>also be required.
>>========================================================================
>>
>>Any further comments? We must be getting close now. :-)
>>
>>Peter
>
>
Received on Fri Feb 7 04:57:07 2003