Re: Calculated data storage?

From: martinl@talk21.com
Date: Thu Jun 28 2001 - 01:22:22 CEST


I concur with the latest additions to this thread. All I would add is that the 'tag' could, optionally, have a form something like:

...
<data type="processed">
 <description>
  Either just a textual description of the how and why the following data has been obtained
or a <subTree> of
       <elements>
          ......describing the processing
        </elements>
      </subTree>
 </description>
   ........
</data>

All the tag names are up for grabs - and anyone can use their own preferred terms (although, of course, any eventual standard will hit on one agreed vocabulary, possibly with tgreed translations in other major caving languages - English (US and UK and Antipodean); Spanish; French; German; Japanese; Russian; Chinese; Malay...?) as XSLT can be used to translate from one tag set to another (the tags could actually be in almost any character set, Cyrillic, Chinese, Japanese included in Unicode, only the angle brackets are required, and a few characters excluded from the possible tagnames).

> Ralph Hartley wrote:
> >Some items that might be contained in the calculated data file are:
> >
> In my proposal uses the Processed element. Maybe "Compiled" is a better
> name, There are lots of other ways to "process" data.
>
>Since the list is a slow as ever I thought I'd pick up on an old thread.
>
>Either "processed" or "compiled" suggests one set of 'obvious' transforms that
>will happen to the 'raw' data. My view of the world suggests that there could
>be any number of processing steps, some of which might even be redundant,
>i.e. I see nothing wrong with a caveXML containing, various versions of
>a line plot. This obviously could be used to ship it somewhere displaying one in
>green one in red for display and discussion about how the various algorithms
>work or don't work! Okay maybe that is an extreme example.
>How about just mulitpass operations?
>I might have code that creates unclosed XYZ values. I want to label that. Why?
>So that I can extract and just that.
>I might have code that does the loop closures. I want to label that. Why?
>So I don't get it confused with the previous data. Personally, like John Halleck,
>I'd make a format that would put these two things right next to each other.
>The file just grows, just like XML allows.
>anyone is welcome to make two version of the file, after throwing away the
>old values if they want to. Personally my master would contain everything
>And I'd XSLT out what I wanted when I have a particular need for some of
>the data.
>
>I might have code that builds rough walls for some simple graphical representation
>of cave. I certainly want to be able to find just these parts in a cave file.
>I might have code that can process a cave XML file and add information
>about which names apply to which points.
>I might have code that analyzes the data and generates extra statistics,
>or inserts cross references to chronologically later surveys,
>or generates 'fancy' models over the data set.
>or cleans up the intersection of various passages.
>who knows what!
>There are so many things you might do to cave survey data.
>Under the assumptions that:
>1. None of these is more important than any other.
>2. I want to keep a caves data all together.
>3. Using physical files to separate different kinds of data is of only
>secondary interest.
>
>I don't see where one of these steps is particularly the 'processed' or
>'compiled' version of the data.
>So my approach would be that for whatever data you are adding to a cave you ought
>provide a tag that adequately defines that particular kind of data,
>steering away from generic terms like 'processed' or 'compiled', or
>'reduced', or 'converted'.
>
>Be specific because we are trying to exchange data. Your 'processed' form hopefully
>is someone else's raw form.
>
>-Paul
>
>
>
>
>
> > - date "compiled"
> >
> > - application used to "compile";
> >
> The date and program attributes, also the version attribute.
>
> >
> > - time/date of "compile"
> > - Options specified for "compile".
> >
> The most common options (process, and treesignificant so far) have thier
> own attributes. The processdetail attribute covers options only the
> processing program could be expected to understand.
>
> > Philosophicly, basicly all the information needed for someone
> > debugging to be able to recreate the "compile".
> >
> I agree to the extent that the format should provide the option to
> record all such information. Of course, there is no way to force the
> compiler to actually save everything. If the CaveXML file is produced
> from the output of the compiler, instead of by the compiler itself,
> there might be a loss of information as well.
>
> > - source file name (CaveXML data file).
> >
> Source location. Remember that it may be in the same file, on the same
> computer, or not. I think that is what XPointer is for. I haven't
> included it in detail in my proposal because I havn't had time to figure
> out XPointer yet, and because XPointer is apparently not a standard, it
> is still only a draft.
>
> >COMPASS has an interesting manner to store the data. Larry uses Move and Draw commands to note the lines. This
> >allows for the drawing of Polylines which are quicker - in most cases - than drawing a series of two point lines. (I don't know
> >if this is also true under Java... yet.) If something similar was taken into CaveXML, it could look something like this:
> >
> > Note that polylines also lend themselves to some optimizations in
> > graphical clipping routines over just two point lines.
> >
> <Line Color="Blue" Size="1>
> > <Station ID="000001" X="0.0" Y="0.0" Z="0.0"/>
> > <Station ID="000002" X="1.5" Y="3.3" Z="-100"/>
> > ...
> </Line>
> >
> >The first Station would be assumed (valid?) to be similar to a move command with all following points/stations draw
> >commands. The ID would tied the station to the CaveXML data file. More data could be copied - station name, etc - or it
> >could be left as a look-up to tie the rest back in. (We have a need to know info about the survey when viewing the map but
> >is it worthwhile to copy all that info into the calcalated data file?)
> >
> My version
>
> <Station reference="000001> <Location easting="0.0" northing="0.0" elevation="0.0"/</Station>
> <Shot>
> <Station reference="000002> <Location easting="1.5" northing="3.3" elevation="-100"/> </Station>
> <Shot>
> ...
>
> If there were no other Station elements with ID "000001" and "000002" (ignoring the fact that IDs can't start with "0"), the reference attribute would be IDs (or could be left offentirely).
>
> A sequence of located stations separated by shots defines a polyline if the program reading the file cares. Any Station in a list like this that is NOT preceded by a Shot element starts a new polyline.
>
> Missing from my proposal is anything like Color="Blue" Size="1". It would be easy to add this to my scheme as attributes of Shot and/or Station (Which could be defaulted), but I have some misgivings.
>
> For a start, so far as I know there are no (or worse many) standard ways to name colors or line thicknesses. What color exactly is "Blue"? Line thickness is 1 what? Pixels? Meters? This is quite a bit trickier than units for cave measuements; there is no universal conversion factor from pixels to meters.
>
> More on this latter in this message.
>
> > (: I suspect this is a topic able to generate a LOT of debate.
> >
> Really?
>
> > At the risk of reserecting a twenty year old discussion... I'll offer one idea:
> > Back when I was actively processing LBCC data, and we had data marked with names
> > (properly scoped, more or less) there was discussion about writing a filter that
> > could display parts of the cave, or produce a file that had only parts of the cave.
> > We invisioned (and never did anything with) commands like:
> > show name "Entrance" and name "Mushroom Room" and not name "Surface"
> > to produce a map that included the entrance, a specific room, removed the surface
> > stuff, and included anything else that fit that window.
> > Since I had code to produce the minimal enclosing window for a section, we could
> > have then produced a plot of just what was of interest. I hasten to add that
> > this was in the days of plotters, and "on screen" graphics was rare... (:Or at least
> > my teletype didn't do them well :)
> >
> Fortunately, I don't need to enter this twenty year old discussion,
> since we are talking about data formats and not about processing or
> display steps. Your application can display anything it wants. We do
> want to be able to preserve the information needed to do this. I will
> think about it.
>
> >Another thing to consider is a view. Again, COMPASS does something similar to this. However, it would also be nice to
> >have the ability to store labels for rooms, passages, and such with the view. At a minimum, the view might contain:
> >
> > - the souce file name(s) (CaveXML data file);
> > - the calculated file name(s) (CaveXML calculated data file);
> > - the view's azimuth, inclination, and dip;
> >
> > - legend text, text style, attributes (scale displayed, north arrow displayed, etc), and location;
> > - label text, text style, and location;
> > - included surveys (allows some survey to be excluded or only specific surveys to be be shown)
> >
>
> And here is where things like color and line thickness belong. They are
> not part of the data gathered in the cave, they are not even properties
> of the cave computedlatter (as the lineplot is), they are instructions
> on how to display the data. It would be nice to separate (conceptually
> if not physically) all the display information from stuff that describes
> the actual cave.
>
> Personally I think display options are going to be hard to define in a
> platform (or even program) independant way. So hard in fact that I don't
> think we should attempt it now. I have to confess that my personal needs
> may be prejudicing my view here; Carto desparately needs a standard
> format for an adjusted lineplot, but has no reason whatsoever to care
> how that data was displayed.
>
> > I.E. Direction of view.
> > As a an old writer of graphics libraries [some still in use!] I would recomend
> > both direction, and the up direction, as a minimum.
> >
> > But, in general, there can be more information. For example, it is possible
> > to apply perspective transformations and other special viewing.
> > My personal preference is to store a (homogenious coordinate) transformation
> > matrix instead of all the information of what transformations are in effect.
> > (But I'll admit that that might not be obvious or comforatable for many, and
> > may therefore be a bad idea.)
> >
> Surely thre are already standards that have all this. VRML must have
> transforms, clips, and just about everything you need to define a view.
> We may want to re-inventthe wheel, but FIRE too?
>
> > By the way, if anyone is interested, I can probably unarchive my (unfortunately
> > ForTran) routines that do rotations, translations, perspective, clipping, etc
> > of coordinates. They just maintain the transformation matrix at each step,
> > and then run the coordinates throught the result and a clipper.
> >
> I have old C code that does just about the same thing, but I would not
> consider using it again myself, never mind inficting it on others. I
> think it was in part of that code that I found a comment that read
> something like:
>
> /* Dear Ralph, You may think you can improve this code. Don't try, you
> will break it. I understand this code better than you ever will. Signed,
> Ralph (date) */
>
> I have found it wise to heed such warnings.
>
> >On the last item, would it be useful to have surveys in the CaveXML data file identified with an ID as well as the stations
> >
> > My personal opinion, yes.
> > But I have a bias to having everything in the file...
> >
> At least as an option yes .
>
> Ralph

--------------------
talk21 your FREE portable and private address on the net at http://www.talk21.com



This archive was generated by hypermail 2b30 : Tue Jul 03 2001 - 00:00:00 CEST