Blog from May, 2011

Work Completed

  • KISTI
    • Investigated how to retrieve results from a finished PTPFlow workflow within cyberintegrator's execution process. I am working through how cyberintegrator works and how we can get cyberintegrator to launch ptpflow workflows and retrieve results that will be ingested into a tupelo backed repository.
    • Waiting on response from Luigi about how to get access to DSE repository so I can browse the code.
  • Cyberintegrator
    • Rob added ContextServer headless to the build and I tested the build. The build worked as expected. We still need to add the ContextServer GUI version to the build and test it.
    • Opened and Resolved CBI-374, login dialog should remember multiple servers. Added a combo box that remembers up to 15 servers, each new server will be saved using the Eclipse infrastructure for saving dialog settings. In the future we may want to expose how many servers get saved and a way of clearing the cache.
    • Resolved CBI-118, create our own import/export/new wizard menu to prevent from getting eclipse additions to these menus. I added a few final changes to this and now the issue is finished. I created import/export categories appropriate for our import/export wizards to provide a slightly cleaner/more organized view of the wizards in the Import/Export menu.
    • Opened CBI-375, delete tool/workflow/data should not be done in the UI thread. There is some lag between deleting an item and the response of the UI and I suspect there is some work being done on the UI thread that should be an Eclipse job to improve UI reponse.
  • Medici
    • Resolved MMDB-1065, changed add tag to use wizard instead of dialog.
    • Resolved MMDB-1145, desktop client throws exception if bean is not available in the context. I changed the log level to warn since there was a comment in the code that the error should not be displayed to the user. A warning seemed sufficient since the most likely cause was not really a major concern (e.g. connected to a different repository).
    • Resolved MMDB-1146, login dialog should remember multiple servers. Added a combo box that remembers up to 15 servers, each new server will be saved using the Eclipse infrastructure for saving dialog settings. In the future we may want to expose how many servers get saved and a way of clearing the cache
    • Reopened and fixed MMDB-839. We changed the GroupedView view name to "Users" with a user icon believing that it only showed user accounts. This was incorrect. This view grouped data by categories specified (by user, by tag) so you could see all the data uploaded by User Foo or with tag Foo1. I changed the name to Grouping and gave it a different icon. We can change this in the future (maybe Data Grouping) since it's old name "Groups" didn't identify clearly what the view was intended for. The grouping is controlled through the views menu with the ability to add additional groupings in the future.
    • Opened issue MMDB-1173 with fix version 1.3. The GroupedView menu should use radio buttons to identify what the current grouping mechanism is. The menu is currently push buttons so it's unclear what the grouping is. Also, the state does not get saved between closing and reopening the application. This can be resolved using radio buttons that save their state.

Work Planned

  • KISTI
    • Continue to work on running a ptpflow workflow job with cyberintegrator
    • Look at DSE code if I can get access to repository.
  • Cyberintegrator
    • Continue working on Cyberintegrator Jira issues.
  • Medici
    • Attend Monday staff meeting (May 23rd)
    • Work on remaining assigned Jira issues and designate issues as fix for version 1.2 or 1.3

Comments

This week went as planned.

Work Completed

  • KISTI
    • Continued working on getting eAIRS to run on the HPC machine Ember since Abe/Lincoln are retired. I was able to successfully run a job using both eAIRS and a really simple PTPFlow Cyberintegrator executor. I added a plug-in that lets a user specify a PTPFlow workflow and then launch the job using the ptpflow engine, which submits the job to an RMI endpoint. I will continue with this work next week to make this more general.
  • Cyberintegrator
    • Opened and resolvedCBI-373, an issue with the command line not properly validating executables if you browse for one. It was not calling validate. This is fixed.
    • Resolved CBI-358, a bug reported that if you create a new workflow and make a tool, the data won't show up properly and the data will disappear. Also, the workflow name was lost. This appears fixed with some updates I've made over the past few weeks. We can reopen if the issue recurs.
    • Resolved CB-36, ability to remove multiple tools/datasets from the tools/data view. I also moved some actions to commands because of a weird UI glitch I was running into mixing commands/actions. It turns out the glitch was caused by the command not having an icon, but the actions are now commands as they should be. This issue does not deal with the implication of what should be done if tools/data are deleted (e.g. should a workflow get updated?) since the data/tool are simply marked deleted. We should open a new issue that defines what the appropriate behavior should be for each view effected.
  • Medici
    • Attended staff meeting. Issues to look into: internationalization with 1 properties file per plug-in, unassigned issues and remaining 1.2 issues)
    • Internationalization with 1 properties file per plug-in deviates from Eclipse best practice, but does not present a problem. We now have a way forward if internationalization is required
    • Users view renaming needs to be reverted (MMDB-839) because it is not a Users view, it is a view that groups data based on menu selection with the ability to add more groupings in the future. This view currently has two menu items, it can show the number of datasets per user or show the number of datasets by tag. We need to come up with a better name/icon for this view.
    • Reopened and resolved MMDB-1141, multiple error popups when disconnecting from a context. The "Users" and "ContextDebugView" do not properly handle disconnecting. This is fixed and the issue is again closed

Work Planned

  • KISTI
    • Work on getting eAIRS to run on the HPC Machine Ember since Abe/Lincoln are retired.
    • Work on getting eAIRS to run using Cyberintegrator launching a PTPFlow job for the eAIRS specific case and then the general case of running a ptpflow workflow.
  • Cyberintegrator
    • Continue working on CI Jira issues, specifically supporting delete for tools/workflows/datasets.
    • Continue working on commenting on the UI and making recommendations for future work.
  • Medici
    • Attend Monday Staff meeting
    • Finish up assigned desktop issues for 1.2

Comments

This week went as planned. I took a personal day on Friday May 20th.

Work Completed

  • General
    • Attended Tupelo code review meeting using google chat.
  • KISTI
    • Started looking at creating a Cyberintegrator executor for PTPFlow workflows.
    • Worked through an issue I had trying to run ptpflow workflows using my renewed grid account/cert with the new HPC machines and PTPFlow. I was getting a "cert expired" error. After working with Al, the cert expired issues is resolved and this week I'll continue working on running eAIRs on different HPC machines since Abe/Lincoln are retired. Once this is finished I can finish my first attempt at a CI executor running a ptpflow workflow job.
    • Reviewed security documents and architecture documents that Jong prepared based on our meeting (May 4th) and his meeting with Jim Basney last week.
  • Cyberintegrator
    • Looked at internationalization with 1 properties file per plug-in. It is possible, but one problem with this is repeated class names. The recommended best practice for internationalization is to use the class name + key for the static string (e.g. MyClass.KEY1 in the properties file); however if there are two MyClass files, then I must deviate from this best practice and do something different. Essentially deviating from one best practice (1 properties file per package) might force you to violate a second best practice (Class name + Key).
  • Medici
    • Attended staff meeting. Tasks include looking over unassigned task list, role enforcement on the desktop and internationalization with 1 properties file per plug-in.
    • Resolved MMDB-1163, remove unnecessary eclipse contributions to the UI. I also created a new category, Debug, for two Bard views (RDF XML and Debug). Those were the last two views listed as questionable in MMDB-845, check that all views are implemented and useful (done by Joel). Those views are useful, but I believe categorizing them as Debug helps the user realize they are more advanced. We can probably close MMDB-845 unless something different should be done with those views.
    • Worked on MMDB-1164, enforce "ViewData" permission. This is mostly done, except that we should decide if the GroupedView should enforce a permission that limits who can view registered user accounts. As an anonymous user, it doesn't seem right that I can see all registered accounts on a server. We probably want to limit this. If not, then this issue is finished.
    • Resolved MMDB-1138, current medici server wasn't displayed. I added this to the status line.
    • Looked over unassigned issues. I updated the assignment/fix versions for issues I knew about or had reported.

Work Planned

  • KISTI
    • Work on creating Cyberintegrator executor to run eAIRS as a PTPFlow tool.
    • Test running eAIRS using the renewed teragrid account. With Abe/Lincoln retiring, we'll need to test to make sure everything is running with different HPC machines and with our renewed account.
  • Cyberintegrator
    • Look at internationalization for CI/Medici projects to see feasibility of 1 properties file per plug-in
  • Medici
    • Attend Monday staff meeting.
    • Work on Medici desktop client tasks. From the staff meeting, there were a few issues I needed to look at (see above).

Comments

This week went as planned. I spent more time this week on Medici to finish off some issues in the desktop client and cleaning up some unwanted UI contributions from Eclipse plug-ins.

Work Completed

  • MAEviz
    • Finished documenting step by step the process of creating a new analysis on the MAEviz wiki in the Documentation section under Tutorials.
  • KISTI
    • Drove to Champaign on Wednesday May 4th, spent the day discussing Cyberintegrator/DSE with Jong/Luigi/Rob. The meeting was very productive and I believe we have a way forward to use those technologies along with PTPFlow in the KISTI project.
    • Worked on and successfully ran eAIRS executable as a command line tool using Cyberintegrator. This should allow us to quickly put together a demo of eAIRS running as a web application using the DSE project.
  • Cyberintegrator
    • Worked on CBI-101, Basic delete is in place for all except step. It's not clear where this should take place and how the various views should respond to delete events. Should workflow with deleted tools be updated? What should they display in the workflow graph view if a tool is marked as deleted? Right now the tools view does not display deleted tools, the data view doesn't display deleted data and the workflows view doesn't display deleted workflows. I have commented about this on the issue for further clarification.
    • Resolved CBI-370. MimeMap was not being initialized properly since a Context was no longer available when Cyberintegrator started. This initialization had to be moved to when a context was set (previously was located inside Activator start method).
    • Resolved CBI-372, a helios bug involving Eclipse p2, exported product files would display an error about the p2 service not being initialized when the application first launched (windows and linux). This can be fixed by specifying start levels for a few plug-ins. The fix is checked in.
    • Added Cyberintegrator text image to login screen to make the login screen more like Medici's and more polished. This was done as part of CBI-363.
    • Resolved CBI-371, updated the about screen with updated links and text.
  • Medici
    • Attended staff meeting
    • Resolved MMDB-1072. Worked with Joe after he implemented an anonymous role on backend server. After a few modifications, we successfully got the anonymous role working on the desktop.
    • Put together wiki page on internationalization options for Medici/Cyberintegrator and future CET projects.
    • Made notes about each remaining desktop issue assigned to me. None are high priority.
      • MMDB-1146 is in regards to the login screen potentially saving more than 1 server address to make it easier to switch between servers. Some thought would have to be given about how this works, how users can clear this out (e.g. if it is saving an erroneous login attempt).
      • MMDB-1145 is one I cannot seem to reproduce again. It involved switching between contexts and an error message popping up if the last selected subject could not be found. I will have to dig a little more and see if I can get this issue to show up or close this issue.
      • MMDB-1065 add tag wizard. This is just cosmetic. We could use a wizard instead of an input dialog to do tagging. It would allow us to tap into the eclipse help mechanism if we wanted to.
      • MMDB-1064 is internationalization. We simply need to pick a method (I documented this on Medici's wiki) and start externalizing strings.

Work Planned

  • MAEviz
    • Finish documenting process of creating a new MAEviz analysis
  • KISTI
    • Meet with Jong/Rob/Luigi on Wednesday May 4th to discuss Cyberintegrator and DSE technologies and how they might be used in the KISTI project
  • Cyberintegrator
    • Continue working on Cyberintegrator issues. Primary focus will be working on delete for tools/datasets/steps/workflows, CBI-101 and getting all views to be synchronized after this occurs and making sure no bugs are introduced. This can be checked in shortly with the understanding that what should happen to certain views is still a work in progress (e.g. does a workflow graph remove deleted tools?).
  • Medici
    • Attend Monday 10am staff meeting
    • Finish Anonymous login for desktop once backend provides role support

Comments

This week went as planned.

Work Completed

  • MAEviz
    • Started on wiki page documenting the step by step creation of a new analysis. This is about 50% done. MAE-1124
  • KISTI
    • Met with Rob, Luigi & Jong via google video chat to discuss eAIRS/KISTI workflow structure and how we can capture this with either the current bean structure or new beans to track provenance. There will be a follow-up meeting this week since it looks like Cyberintegrator/DSE could be a good fit for our problem.
    • Looked a BlobBean, a suggested bean type that could allow us to associate more than 1 blob with a bean.
    • Started working on wiki page that lays out the eAIRS problem in more detail for the Wednesday meeting to make sure everyone understands our needs so we can determine if DSE/Cyberintegrator will be a good technology fit for our project. I will continue to work on this for the coming meeting so the problem scope is clear.
    • Briefly went through KISTI's eAIRS UI Mockup. I will continue this before Wednesday's meeting so we can discuss the UI and required tasks.
  • Cyberintegrator
    • Added more comments to Cyberintegrator UI review
    • Added a comment to CBI-111 (improve Info view) about including more workflow information. It wasn't clear what Luigi was information he was referring to so I asked for clarification. I have added more info for datasets and tools (e.g. created by, date, etc), but I don't see what else can be displayed for workflows.
    • Worked on CBI-101, I added delete handlers for datasets, workflows and tools. I am still testing everything to make sure all views are in sync after a tool/dataset/workflow gets deleted.
    • Resolved/Closed CBI-126, password in context login is visible. This does not appear to be relevant. The login screen does not display the password field.
  • Medici
    • Added a comment to Medici UI review. If an image preview is not created (e.g. an image was ingested using a different tool), there is no way to view the image information inside Medici (e.g. InfoView) because it expects there to be a preview available.

Work Planned

  • MAEviz - add wiki page that documents how to create a new MAEviz analysis step by step
  • KISTI - Meet with Rob and Luigi to discuss using/creating Tupelo beans to capture provenance for KISTI workflows
  • Cyberintegrator/Medici
    • Work on Cyberintegrator issues, including CBI-111 (improve info view) and CBI-101 (allow to delete tool/data/step/workflow).
    • Continue working on CBI UI review

Comments

This week went as planned. Next week (week of May 2nd) I will be visiting Champaign all day Wednesday.