From Federico Carminati
To David Adams
September 9, 2003


Dear David,
   thanks for your answer. 

On Mon, 8 Sep 2003, David Adams wrote:

[...]


>> If so, I believe we have a similar model: you speak of datasets and
>> Datacollections where I speak of files (or tape records or ...) and
>> datasets. I prefer to reserve dataset for the higher level concept.


This is partly the source of the problem. It has been the agreement in
HEPCAL to abandon the term file and use dataset instead, which is very
close to a "directory" in Unix, i.e. a name that indicates a set of files
or a set of datasets.  In HEPCAL a dataset is a directory that EITHER
contains only files OR cointains only directories.


>> Are we reduced to the point of only disagreeing on the name, 
>> Datacollection or dataset? Or am I still missing an important point?


Yes and no. The point is that I believe that the higher level concept that
you are defining (Dataset for you, datacollection for me) should be built
on a lower level entity (file for you, dataset for me) distinct from it
and with well defined functionality. The issue here is also the definition
of requirements and the place where functionality is provided. I agree at
the end one will need something like the functionality you describe. The
question is how to get there. Current grid products are far away from your
requirements. The HEPCAL choice was to describe the "minimal"  
functionality that could be used. Suppose you have a grid that handles
datasets (in my definition, but we understand each other now) in a
production-grade way. This *is* doable and indeed, to my knowledge, at
least AliEn is very close to it. Then you can build your additional
functionality on that, delegating to the lower level the jobs of
optimising access and dereferencing logical names.

Hope this helps. Fed