Requested features for ADA/DIAL
Updated 20oct05

1. Use DQ for replica selection and file transfer.
-- Use DQ2.
-- For replica selection, use Pool FC interface.
-- File transfer is probably gsiftp for single files but we should
   check if FTS will provide option for interactive response.
-- For release 1.40.

2. Thread safety for catalogs.
-- What does this mean?
-- Threads are less of an issue because the analysis service does
   little parallel processing in DIAL 1.20.
-- Consider this done in 1.20.

3. WS scheduler should put all job def objects in catalogs for memory
   management and provenance tracking.
-- Insert app, task and dataset in WS scheduler imp.
-- Insert result in CompoundJob as soon as job finishes.
-- Require this 1.30.
-- Done in 1.21.09.

4. Provide easy retrieval logs for tasks and jobs.
-- Proposal: Tar up run directory for each job and provide means for
   user to fetch this directory structure
-- Interface: root> msch.job(jid).open_archive()
-- Will handle tasks by treating their build as a job.
-- Require this 1.30.
-- Done in 1.21.09.

5. Grid-compatible means for building and retrieving tasks.
-- Proposal: Make task build a subjob of CompoundJob.
-- This will make task build non-blocking.
-- Require this for 1.30.
-- Done in 1.21.10.

6. Add app/task help() that retrieves readme.txt.
-- Require this for 1.30.
-- Done in 1.21.09.

7. Add TextDataset.
-- Done in 1.20.

8. Add ScriptedJob.
-- Done in 1.20.

9. String queries for PyDIAL.
-- SqlQuery provides automatic conversion from string and const char*
   and provides method to_string(). These should suffice.
-- Do not take any further action.

10. String job status for PyDIAL.
-- Add Job::status_as_string.
-- Done in 1.21.11.

11. Add different strategies for failed subjobs and retries for
    subjob failures.
-- Defer to 1.40.

12. Add means to eliminate sick nodes for LSF, Condor, ....
-- Defer to 1.40.

13. Create Rome ESD datasets
-- Consider using DQ definitions.
-- Defer to 1.40.

14. Understand how to access ATLAS DM datasets.
-- When will Rome AOD or ESD datasets be available for us?
-- Define Dataset subclass(es) implemented with DDM catalogs.
-- Require this in 1.40.

15. Common atlas-dial package to be shared by all ATLAS applications
-- Called atlasapps.
-- Done in 1.20.
-- Tadashi working on documentation.

16. Cache LSF status to reduce # of bjobs calls.
--Done in 1.20.

17. Wrapper for prodsys transformations.
-- Christian working on this.
-- Want this for 1.30.
-- Defer to 1.40.

18. Enable users to write to catalogs.
-- Done in 1.21.07 with WS's deployed at BNL

19. Add plugin handler.
-- Command dataset_property does not work for WS catalogs.
-- Patch in 1.21.08 fixes dataset_property by loading libraries in
   GenericRepostiory and GenericSelectionCatalog.
-- For 1.40 or later.

20. Enable transformations atlasopt, atlasdev and atlasdev-src to
    handle data production.
-- Need means for user to provide content label (ESD, AOD,...).
-- Add output_content to tasks to carry this information and the
   mapping to output file names.
-- Have example task for atlasopt esd2aod.
-- Require this 1.30.

21. Deploy service to directly submit jobs to CE
-- Using Condor-G or globus gatekeeper.
-- Investigte using ScriptedJob.
-- For Hyunwoo?
-- For 1.30?
-- Look at this after 1.30

22. Develop and deploy service to distribute job over other
    analysis services.
-- Defer to 1.40.

23. Deploy service to distribute jobs using LCG/gLite WMS.
-- Dietrich is working on this.
-- Like to have stable, continuous service for 1.30.
-- Farida will take this over.
-- Defer to 1.40.

24. Make dataset hierarchy fast and scalable.
-- Separate out content and event ID lists.
-- Not for 1.30. May be moot or part of ATLAS dataset integration.

25. Make transformations easy to run by hand.
-- Like this for 1.30.
-- Defer to 1.40.

26. Create ADA user guide.
-- On ADA web pages rather than DIAL release page.
-- Change this to DIAL user guide on DIAL release page.
-- Done for 1.30.

27. Deploy service to submit and monitor jobs in prodsys.
-- Frederic is working on this.
-- Require for 1.30.
-- Defer to 1.40.

28. Implement APS executor using DIAL services.
-- USATLAS PANDA project.
-- Not for 1.30.
-- Require for 1.40.

29. Add job persistency to analysis services.
-- Done in 1.21.08.

30. Analysis service running continuously at UTA.
-- For 1.30?
-- Defe this until after 1.30.

31. Service selection catalog to help users and selection service
    choose service endpoint. Also use this for monitoring.
-- Initial implementation in 1.30.

32. Analysis service using PANDA pilot job mechanism.
-- For Hyunwoo?
-- Use ScriptedJob?
-- Require this in 1.40.

33. Integrated graphical service/job monitor.
-- Starting from list of services, user can drill down to jobs and
   subjobs and then to results and archives.
-- Base on DIAL catalogs.
-- Service web page in 1.30 is a start.
-- For 1.40.

34. Add modification time to job repository.
-- To help in monitoring.
-- Done in 1.30.

35. Make JobPreferences a map of strings.
--Done in 1.21.11.

36. Add CPU time and memory to Job.
--Defer to 1.40.

37. Web page to document and monitor service instances.
--Requires two ports for GSI and noGSI.
--Done in 1.30.