A use case from the National data Service (NDS)

Here’s one from EPOS
[cid:***@***.***]
Keith G Jeffery Consultants
Prof Keith G Jeffery
E: ***@***.***
T: +44 7768 446088
S: keithgjeffery
Past President ERCIM www.ercim.eu (***@***.***)
Past President euroCRIS www.eurocris.org
Past Vice President VLDB www.vldb.org
Fellow (CITP, CEng) BCS www.bcs.org
Co-chair RDA MIG https://rd-alliance.org/internal-groups/metadata-ig.html
Co-chair RDA MSDWG https://rd-alliance.org/working-groups/metadata-standards-directory-work...
Co-chair RDA DICIG https://rd-alliance.org/internal-groups/data-context-ig.html
----------------------------------------------------------------------------------------------------------------------------------
The contents of this email are sent in confidence for the use of the
intended recipient only. If you are not one of the intended
recipients do not take action on it or show it to anyone else, but
return this email to the sender and delete your copy of it.
----------------------------------------------------------------------------------------------------------------------------------
- Show quoted text -From: gbergcross=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of Gary
Sent: 21 October 2014 22:21
To: Data Fabric IG
Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
Using the NDS: a scenario
To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
--
Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
Manage my subscriptions: https://rd-alliance.org/mailinglist
Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an ‘external’ datapoint when considering the datafabric.
Rob
From: Gary <***@***.***>
Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
Date: Tuesday, October 21, 2014 at 11:21 PM
To: Data Fabric IG <***@***.***-groups.org>
Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
Using the NDS: a scenario
To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
--
Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
Manage my subscriptions: https://rd-alliance.org/mailinglist
Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
Best
peter
From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
Sent: Wednesday, October 22, 2014 12:05 PM
To: ***@***.***-groups.org
Cc: Pennington, Robert Lee
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an 'external' datapoint when considering the datafabric.
Rob
From: Gary <***@***.***>
Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
Date: Tuesday, October 21, 2014 at 11:21 PM
To: Data Fabric IG <***@***.***-groups.org>
Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
Using the NDS: a scenario
To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
In 2021 the LIGO gravitational wave observatory detects a strong "transient" burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
--
Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
Manage my subscriptions: https://rd-alliance.org/mailinglist
Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

I can tell more about the virtual observatory (the disciplinary interoperability layer), and on how we astronomers deal with information found in published papers if useful.
Cheers
Francoise
Envoyé depuis mon appareil Android.
-----Original Message-----
From: Peter Wittenburg
<***@***.***>
To: "***@***.***-groups.org" <***@***.***-groups.org>
Sent: mer, 22 oct. 2014 14:06
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
Best
peter
From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
Sent: Wednesday, October 22, 2014 12:05 PM
To: ***@***.***-groups.org
Cc: Pennington, Robert Lee
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an 'external' datapoint when considering the datafabric.
Rob
From: Gary <***@***.***>
Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
Date: Tuesday, October 21, 2014 at 11:21 PM
To: Data Fabric IG <***@***.***-groups.org>
Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
Using the NDS: a scenario
To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
In 2021 the LIGO gravitational wave observatory detects a strong "transient" burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system,
containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
--
Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
Manage my subscriptions: https://rd-alliance.org/mailinglist
Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

Yes would be very good to have short notes that help us how the "Data Fabrics" in your realm are currently implemented.
Peter
- Show quoted text -From: francoise.genova=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of Francoise Genova
Sent: Wednesday, October 22, 2014 2:16 PM
To: ***@***.***-groups.org
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
I can tell more about the virtual observatory (the disciplinary interoperability layer), and on how we astronomers deal with information found in published papers if useful.
Cheers
Francoise
Envoyé depuis mon appareil Android.
-----Original Message-----
From: Peter Wittenburg
<***@***.***>
To: "***@***.***-groups.org" <***@***.***-groups.org>
Sent: mer, 22 oct. 2014 14:06
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
Best
peter
From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
Sent: Wednesday, October 22, 2014 12:05 PM
To: ***@***.***-groups.org
Cc: Pennington, Robert Lee
Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an ‘external’ datapoint when considering the datafabric.
Rob
From: Gary <***@***.***>
Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
Date: Tuesday, October 21, 2014 at 11:21 PM
To: Data Fabric IG <***@***.***-groups.org>
Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
Using the NDS: a scenario
To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
--
Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
Manage my subscriptions: https://rd-alliance.org/mailinglist
Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803