Blog

Work Completed

• Worked on KISTI Project and resolved the following issues:
  1. KNSG-55, updated wiki tutorial to include a section on running the parameterized workflow
  2. KNSG-57, added an about screen to the software under Help to display version and contact information
  3. KNSG-59, added imported results from the remote launch machine to the users local repository so they can be accessed within the application, possibly by post processing software that can be launched from our application.
  4. KNSG-61, which allows users to add annotations to datasets (comments, notes, etc)
  5. KNSG-62, when retrieving datasets from the remote launch machine, added an option to tag the dataset as a result to the user interface
  6. KNSG-63, RMI page was not displaying if no RMI services were running, it now displays with a error message that the user needs to define a service before launching jobs
  7. KNSG-64, added popup menu option to add a tag to a dataset
  8. KNSG-65, added the New Scenario menu option to the Scenarios view, it was previously only available from the File menu.
  9. KNSG-66, after a workflow is setup, it now is displayed as part of the Scenario so it can be resubmitted.
  10. KNSG-67, as an improvement, the user interface for adding the gridftp service was simplified into required and optional fields
  11. KNSG-68, updated the default myproxy URI to be the teragrid myproxy run by NCSA so KISTI users could use Teragrid accounts to use the software.
  12. KNSG-71, added documentation to the wiki on where the eAIRS CFD code should be installed and how to install it, a requirement for running the workflows.
• Released version 0.5 of the software on Friday August 20, 2010, added release notes the wiki.
• Briefly tested the latest Marketmaker for bugs and reported my findings to Jong.

Work Planned

• Work on KISTI project and release version 0.5 this Friday August 20, 2010. Update the wiki to document the latest release.

Comments

This week went as planned. Next week I will continue working on eAIRS

Work Completed

• Worked on KISTI Project
  1. Finished KNSG-49, users can now add gridftp information for machines so that the information can be used to auto-populate the wizard page for retrieving result files. It saves the user from entering the machine URI each time results are to be retrieved, an operation that is prone to error since the URI must be exactly right.
  2. Finished KNSG-54 and KNSG-56, after selecting the finish button for an analysis wizard, there was a long delay between the button being pressed and the execution taking place and there was a check occurring to see if an RMI extension point was available to launch the job. I added a page for the user to select the RMI extension (related to PTPFLOW-16) and altered the finish operation to show a busy cursor to indicate progress was occurring.
  3. Worked on KNSG-59, results that are selected for retrieval from remote machine that job ran on should be imported into the repository and added to the users scenario, this is nearly finished and requires testing.
  4. Finished KNSG-60, added a view to display properties associated with a dataset, including tags.
  5. Worked on KNSG-61 which involves allowing the user to add annotations to datasets (comments, notes, etc), this is nearly finished and will be completed next week.
• Worked on PTPFLOW
  1. PTPFLOW-16 - added an error message to RMIRefreshClientWizard if no RMI endpoints exist, otherwise a user is unaware that no RMI services have been defined. I also updated how the wizard page creates its drawing container because the specified dimensions were causing problems if the page was reused in another wizard. I tested the fix and no problems were noticed.
  2. Contacted Albert Rossi about a possible API change for Trebuchet, he is out of town and will look into my issue when he returns. He provided me some possible workarounds that I will test this coming week.

Work Planned

• Work on KISTI project
  1. KNSG-49
  2. KNSG-54 and KNSG-56
  3. KNSG-59
  4. KNSG-60
  5. KNSG-61

Comments

This week went as planned. I took a personal day on Thursday August 12, 2010.

Work Completed

• Reviewed KISTI/MarketMaker text for a meeting Jong had with Jim and NCSA management
• Finished Performance Review for 2010-2011 and sent to Jong.
• KISTI Project Bugs and Features
  1. Completed KNSG-50 - error creating multiple directories on the machine executing the job. This is fixed.
  2. Completed KNSG-53 - add analysis pages more parameterized CFD workflow, this is finished, but needs to be tested. I cannot test from my machine with the RMI service being a real host, PTPFlow communicates back to the launching machine and my host is not real behind my router. The RMI multi-user service will solve this issue.
  3. Completed KNSG-52 - After a job completes, we need to get the results back to the users machine. This is finished. I created an SSH-like interface for browsing the launch machine and pulling data back to the users machine.
  4. Released and tagged version 0.4, updated the project status on the wiki, added new Tasks/Features to Jira and updated Tasks that have been moved to the next release, including setting up the build server for automated builds and setting up an RMI service for multi-user support. These tasks were moved because we were waiting on a machine to be configured.

Work Planned

• Work on KISTI project and Release version 0.4 of KISTI project

Comments

This week went as planned. I took a personal day on Friday August 6, 2010.

Work Completed

• KISTI Project
  1. Finished KNSG-48, search for input dataset. There is now a simple way to search for input datasets for the CFD analysis and any new analyses added.
  2. Worked on KNSG-52, after a job completes we need to retrieve the result datasets from the server through the RCP application using the clients MyProxy and gridftp. This is nearly complete and just requires some testing.
  3. Added some icons for commands that were missing icons (e.g. retrieve dataset and install sample dataset).
  4. Helped Al with displaying commands for a view based on the perspective that is active.

Work Planned

• Work on KISTI Project, the CFD analysis page needs an option to search for input datasets that fit the field types and after running an analysis, we need some way to retrieve the output datasets from the server that ran the job.

Comments

This week went as planned. Next week I hope to work with Jong and Bruce to configure a machine that Jong acquired to run a dedicated RMI service for launching jobs and to start setting up a build server to run daily builds. I took half of a personal day on July 30, 2010.

Work Completed

• KISTI Project
  1. KNSG-41 - Add wiki tutorial page for eAIRS RCP. This first cut is finished and will be polished this week from comments Jong provided Friday afternoon.
  2. KNSG-43 - investigate moving files from the users machine to a remote machine programatically using the myproxy certificate. This feature impacts the ability for the user to specify local data that should be used in the workflow executed remotely. The feature is implemented with some more future work possible (e.g. we use gridftp, there are other methods for moving data we could support).
  3. KNSG-46 - create simple user interface for specifying inputs to eAIRS workflow. This is finished.
  4. KNSG-47 - users should be able to tag datasets. This feature is finished and allows user input fields to be populated based on dataset tags.
  5. I helped Al Rossi test the multi-user feature of PTPFlow that he has reimplemented. We debugged issues that I had and successfully got this feature working.
• Tagged and Released Version 0.3 of eAIRS RCP

Work Planned

• Work on KISTI project and release version 0.3 on Friday July 23, 2010. Release 0.3 will have a simple UI for setting up the workflow, moving the user input data to the machine that will execute the workflow and then executing the workflow.

Comments

This week went as planned. Next week I will be working on setting up the parameterized workflow and discussing with Jong the current status of the project to obtain feedback on the current feature set.

Work Completed

• KISTI project
  1. KNSG-42 Added menu option to the repository to add a dataset to a scenario
  2. KNSG-44 - added remove dataset option and a specific handler for repository (delete) and the scenarios view (just remove).
  3. KNSG-43 - investigate moving local data to the remote server where a job is being launched. This is a work in progress.
  4. KNSG-46 - create a simple UI for specifying user inputs and launching a workflow job. This is a work in progress.

Work Planned

• Work on KISTI project, specifically setting up the infrastructure for launching a workflow through a simple UI that allows users to specify the inputs, transferring the data over to the server where the job is going to be launched and using that data in the workflow. The next step will be to specify imported data from the user's repository. The tests only included browsing for data locally.

Comments

This week went as planned. I took a personal day on Friday July 16. Next week I will continue working on some of the above features and I will start working on a tutorial wiki page to include with the release.

Work Completed

• KISTI Project
  1. KNSG-33, KNSG-34, KNSG-35, KNSG-36, the major issue was KNSG-33 involving simple file import as we move towards putting a MAEviz-like interface on the HPC workflows allowing users to specify local files that are to be staged when the workflow is submitted. File import is now in place and can be viewed in the repository.
  2. KNSG-37 - I updated PTPFlow dependent plug-ins to match Al's refactoring of PTPFlow. Some packages were changed.
  3. KNSG-38 - added drag and drop support for adding files to a scenario.
  4. KNSG-39 - added a separate view for seeing which workflows are in the repository.
  5. KNSG-40 - added a sample dataset to the 0.2 release that is installed on the users machine.

Work Planned

• Work on KISTI project and release version 0.2 on Friday July 9.

Comments

This week went as planned. Monday was a holiday and I took a personal day on Friday July 9. Soon we will need to start working on a wiki page for using the releases as more features get added and the software becomes more usable.

Work Completed

• KISTI
  1. KNSG-18 - updated wiki developer setup for the Eclipse 3.6 Helios release
  2. KNSG-32 - when logging into eAIRS, added the option to save the users password in the interactive login screen
  3. KNSG-26 - added option to let the user enter a MyProxy server URI
  4. KNSG-33 - import file to a repository. This is a work in progress.
• PTPFlow
  1. PTPFLOW-13 - updated UI actions in ncsa.eclipse.pwe to use the commands framework. There were some complex actions that allowed this to be a nice example of how to us org.eclipse.ui.services to have one button activate when another one deactivates (e.g. like play/pause).

Work Planned

• Work on KISTI Project, we need a minimal import process to import data into eAIRS so we can create a simple UI for setting up and launching workflows to the supercomputers using PTPFlow's RMI service. This process will mimic a lot of the UI that was created for MAEviz to give users a simple UI for launching jobs.
• Work on improving PTPFlow UI's that are relevant to the eAIRS project by converting actions to the Eclipse commands framework.

Comments

I took personal time off Thursday afternoon (1/2 day) and all of Friday (1 full day). This week went as planned.

Work Completed

• KISTI Project
  1. Resolved KNSG-20, added right click menu to workflows for bringing up the submission dialog.
  2. Resolved KNSG-21, added interactive splash handler that lets the user log into MyProxy or work offline.
  3. Resolved KNSG-21, added browse button to preference page for specifying PTPFlow RMI distribution directory.
  4. Resolved KNSG-24, eAIRS product should depend on and build from features. I Defined features for each set of plug-ins (PTPFlow RMI, KNSG framework, eAIRS) and tested the product using those defined features.
  5. Resolved KNSG-25, added sample workflow that is installed when eAIRS first launches.
  6. Resolved KNSG-28, added application icon for eAIRS (16x16, 32x32, 48x48, 64x64, and 128x128)
  7. Resolved KNSG-30, added refresh button to repository view so users can refresh the repository content.
  8. Resolved KNSG-31, added a help menu to eAIRS.
• Built version 0.1 with Eclipse 3.6 for Win32, Linux x86 and x86_64, tested on win32 and linux x86. Released version 0.1 of KISTI project on Friday June 25, 2010.
• Started a google document outlining the features for upcoming releases, only version 0.1 was completed, 0.2 still needs work.
• Setup KNSG Jira site to include more information when adding features such as which release they belong to and which component for better tracking.

Work Planned

• Build, Test and Release version 0.1 of KISTI Project on Friday June 25, 2010

Comments

This week went as planned. I took a personal day on Friday (finished release and provided Jong with builds for release). Next week I intend to continue working on the KISTI project, we will work on what will be in release 0.2, and the build still needs to be automated with Hudson.

Work Completed

• KISTI Project
  1. KNSG-16 - created a wiki page for setting up new developers development environment, including a single team project set that will automatically check out any missing plug-ins. Al did the same for PTPFlow plug-ins. Users would still require SVN access to the repositories defined in the team project set.
  2. KNSG-19 - created a base abstract handler class for use with commands that use the commands framework. This base class contains the boiler plate functionality. All PTPFlow command handlers also use this class.
  3. KNSG-17 - defined a double click listener to open workflows in PTPFlow's XML Editor View.
  4. KNSG-15 - add a button to the scenario view to remove workflows from a scenario.
  5. PTPFLOW-1, PTPFLOW-2, PTPFLOW-3, PTPFLOW-4, these all involved moving UI specific code outside of non-ui plug-ins for a clean data model. This code was checked in this week and verified with testing that nominal functionality all works. Some minor bugs could remain where any hard-coded plug-in package names slipped through.
  6. PTPFLOW-9 fixes a bug where a package name was incorrect.
  7. PTPFLOW-10 - WorkflowBuilderModel had popup menu items defined with the old actions framework. I converted this to the commands framework so we have more control over whether these actions display and in what menus.

Work Planned

• Work on KISTI project, continue to separate PTPFlow's UI functionality from the model code, updated old actions to the commands framework where necessary to provide better defined actions that users have more control over where they are placed and when they show up.

Comments

This week went as planned.

Work Completed

• Worked on KISTI project
  1. PTPFLOW-1 - move UI outside of plugins defining data models
  2. PTPFLOW-2 - subtask of PTPFlow-1, move UI pieces outside of ncsa.eclipse.repository into ncsa.eclipse.repository.ui. This was the core plugin for a lot of PTPFlow's plugins. It tooks some time to fix all of the broken parts that were depending on the UI pieces.
  3. KNSG-11 - added scenario view, similar to MAEviz scenario view, where users can add workflows and work with them.
  4. KNSG-8 - added drag and drop to scenario view for workflows. The RCP client can now tap into PTPFlows launch mechanism and launch workflows from the scenario view.
• Tested preliminary data model changes to PTPFlow and everything seems to be working properly. I hope to finish the migration of UI bits to UI plugins this week, test the changes and then check it back into Al's repository.
• Worked on Marketmaker project
  1. The selection of census tracts was not allowing unselection through ctrl+click, instead it was selecting the feature again. Jong and I narrowed this down to new feature Id's being generated with each selection so OpenLayers believes a new selection has occurred. The selection talks to a geoserver backed by a postgres database so we must determine why the selection doesn't return the same feature id.

Work Planned

• Work on KISTI project, we need to separate the UI components better in PTPFlow. Currently, UI and non-UI bits (models) are within the same plugin. This will cause issues later with any web based application for KISTI. I discussed my suggested changes with Al Rossi before proceeding and have kept him up to date on what I'm doing.
• Work on Marketmaker project

Comments

I intend to continue working on PTPFLOW-3 and PTPFLOW-4 which are also subtasks of PTPFlow-1, to remove the UI dependencies in the data model so that our KISTI plugins that define new models don't depend on UI plugins. This week went mostly as planned. I had intended to finish more of the UI migration, but was sidetracked by the census tract bug in Marketmaker. With it's release upcoming, it took priority.

Work Completed

• Worked on KISTI Project
  1. Requested a Jira Project on the CET Jira site for PTPFlow so that PTPFlow fixes can be tracked there and so we can see what features Al is working on
  2. Fixed some PTPFlow issues, refreshing the RMI client failed with null pointer if the Service Registry view was never opened, this is fixed. Dual Monitor had an issue where the login screen displayed between monitors, this is fixed.
  3. Added some preferences for staging PTPFlow pieces so that PTPFlow could be installed from an RCP application, the current version expected a feature to be installed in eclipse that contained the PTPFlow parts.
  4. Started converting some of PTPFlows menus and actions to the Eclipse commands framework to allow for more control over reuse and placement of menus since it is more declarative.
  5. Started working on the KISTI projects version of PTPFlows repository and scenarios view so that we can use a MAEviz like analysis view to launch jobs on the supercomputers using PTPFlow.

Work Planned

• Work on KISTI Project
  1. Start developing RCP application using pieces from PTPFlow
  2. Update parts of PTPFlow so we can reuse some of PTPFlows views and plugins

Comments

This week went as planned. I talked with Jong briefly and we will discuss his KISTI visit and the features they would like to see developed over the new few months.

Work Completed

• KISTI Project
  1. After the meeting in Champaign, we've determined that the semantic capabilities will be a phase II feature. We came away with a basic project plan for Phase I deliverable to KISTI (an improved, RCP version of PTPFlow that they could install as an application) and Phase II would start moving towards the usage of Beans.
  2. Created an updated Mockup of the KISTI RCP Application based on information from the meeting. I also updated the wiki with some text around the new mockup.
  3. Started working on adding some PTPFlow views to the RCP application. I talked with Al about updating the actions to use the Eclipse commands framework, a more declarative way of adding actions and menus. I will start converting a few of the PTPFlow menus and actions to the commands framework so that Al, Jong and/or myself can later update more of the PTPFlow actions to use the commands framework.
  4. Still working on getting my parameterized workflow to run successfully, it's a simple "echo" message workflow that should echo a message 4 times, but it fails. At this point we're not sure if it's the fact that my machine is behind a router with no real IP address or not. The single run workflow works. Al can run it successfully, Jong did not get a chance before leaving and will try to get it working with Al when he returns. I will continue to work on the RCP application in the meantime.

Work Planned

• Meeting in Champaign to discuss how to integrate semantic features into the KISTI project.
• Work on KISTI Project

Comments

I travelled to Champaign on Monday May 24 and I took a personal day on Friday May 28.

Work Completed

• Marketmaker
  1. Finished MM-165, I wrapped the WMS Post class of OpenLayers for gwt-openlayers that allows the use of HTTP Post.

• KISTI
  1. Put together documentation on the wiki site outlining a possible data model for the project, including the usage of tupelo beans and views, some of it based on Al's PTPFlow and some possible improved views.
  2. Started to create the base plugins for the KISTI project (RCP, UI, Core and test plugins).

Work Planned

• Work on the KISTI project
• Work on Marketmaker project

Comments

This week went as planned. Next week I will be meeting with Rob, Luigi, Al and Jong to discuss using tupelo/bard for KISTI and some data scenarios we will encounter using the HPC machines.

Work Completed

• KISTI
  1. Worked on submitting parameterized workflows with PTPFlow, I sent Al Rossi my workflows and error logs because I was having issues with using the parameterized option. I made some progress, but Al will need to look into some exceptions I was getting.
  2. Made some minor edits to the KISTI storyboard for and we submitted for Dr. Nam's review so we can move forward.

• Marketmaker
  1. Closed duplicate issues
  2. Tested Marketmaker with the latest version of OpenLayers for bugs (v2.9.1)
  3. MM-146 - CSS file had broken entries, this is fixed
  4. MM-149 - Merged the latest changes/fixes from the GWT-OpenLayers project with our version of GWT-OpenLayers
  5. MM-151 - Jong resolved this issue, the Advanced tab was next to the Search tab and it appeared as the advanced options of the search, when it was the layer controls, I provided feedback and some suggestions.
  6. MM-152 - The background layer control was out of date with what was being displayed, google street maps were displayed, but the control said hybrid was.
  7. MM-153 - The Navigation toolbar initially did not display in Chrome browser, although it displayed properly in other browsers. This is fixed.
• Marketmaker 0.5 beta was released and upgraded to 2.9.1 of OpenLayers, I cam currently working on an issue that appears only in IE-8 where selection of census tracts does not clear properly, MM-156.

Work Planned

• Work on KISTI project
• Work on MarketMaker project with a planned release of 0.5 beta on Friday May 14, 2010.

Comments

This week went as planned, I will continue to work on the Marketmaker bug MM-156, test the new release and work KISTI next week after our storyboard has been reviewed.