 |
|
|
|
A Scalable, Open Source Platform for Data Processing, Archiving and Dissemination
|
|
|
Description
|
End users have a lot of data, but do not have the expertise needed to analyze it. The goal of this project is to empower end users to analyze big data by demonstrating that: 1) data analytics experts can use open source software to quickly assemble workflows, 2) end users can easily run these expert-grade workflows and get useful views on their data. Our work combines semantic workflow capabilities of the WINGS workflow system with scalable data systems and workflow execution infrastructure available in the OODT framework. Our work includes a release of the integrated system as open source software within the Apache OODT project.
More-->
|
|
Status
|
An important need for users of big data systems is to get an answer within a specified timebound. A current limitation of workflow systems is that they do not take into account user deadlines: they run workflows selected by the user, but take their time to do so. This is impractical when large datasets are at stake, since users often prefer to see an answer faster even if it has lower precision or quality. We recently developed WOOT, an extension to workflow systems that enables them to take into account user deadlines by automatically generating alternative workflow candidates and ranking them according to performance estimates. The WOOT system makes these estimates based on workflow performance models created from workflow executions, and uses semantic technologies to reason about workflow options. Possible workflow candidates are presented to the user in a compact manner, and are ranked according to their runtime estimates. WOOT combines and extends capabilities from the WINGS semantic workflow system and the Apache OODT Object Oriented Data Technology and workflow execution system.
More-->
|
|
Research
|
More -->
|
|
Publications
|
- Time-Bound Analytic Tasks on Large Datasets through Dynamic Configuration of Workflows, Yolanda Gil, Varun Ratnakar, Rishi Verma, Andrew Hart, Paul Ramirez, Arni Sumarlidason, Sam Park. In Proceedings of the Eighth Workshop on Workflows in Support of Large-Scale Science (WORKS), held in conjunction with ACM Supercomputing 2013, Denver, Colorado, November 2013
- Capturing Data Analytics and Visualization Expertise with Workflows, David Kale, Samuel Di/person> Yan Liu, Yolanda Gil. In AAAI Fall Symposium in Discovery Informatics, November 2013
- Large-Scale Multimedia Content Analysis Using Scientific Workflows, Hyunjoon Jo, Ricky Sethi/person> Andrew Philpot, Yolanda Gil. In ACM International Conference on Multimedia, October 2013
More publications -->
|
|
Demo
|
See a demo -->
|
|
People
|
Group Members:
More -->
|
|
Funding
|
- US Air Force Office of Scientific Research (AFOSR)
- DARPA
More -->
|
|
Links
|
More links
-->
|