Doug Fein Need to remind LSST and other Nebula users to attend these meetings Nebula has been running for 1 year 2 months Incredibly stable environment More than 10 million virtual CPU hours Started as 10 compute nodes Upwards of 60 compute nodes now 12 storage nodes some used for experimentation Experiments ITS has been working on Lin Cheng is a graduate student who has been working with OpenStack for ITS. He has installed a test version of Mitaka and will be working on installing Newton in October when it comes out. ITS is looking at docker containers and shared authentication (LDAP) Our goal is to upgrade OpenStack on Nebula between 12/17 and 1/10 We hope to have no downtime of services We plan to bring up a secondary environment and migrate from one to another. Testing moving forward. We need everyone's instances to be in the migratable portion of Nebula. All instances started since the last meeting are on the live migration portion of Nebula. GlusterFS is taking a portion of every storage node (6TB), glance images sit on it. GlusterFS is mounted onto every live migration node, does mem copy, keeps instances in the shared FS. If there is a node or a network issue we can move instances away from the problems with no downtime to the users. It will allow a live migration to a new version of OpenStack. Chris have been sending e-mails to everyone who has instances on the old nodes that need to be cleared. You will get a note, schedule time to shut it down and restart it on live migration node. Working on changes on how we communicate to users about their instances. PI of project and instance owner will now get notifications. We are filling a lot of the resources on Nebula >50% and < 90% of virtual CPUS are in use. Making decent use of the RAM Under 5% usage on the storage One minor issue in last couple months - a few compute nodes weren't getting all of their GRE tunnels mapped to them. It is a known issue that networking has to be reworked when OpenStack gets past a couple hundred nodes. We put in place monitoring and an automated fix of the GRE tunnels. Security incidents have started to occur on Nebula Open ports, intrusions due to default passwords The security group is doing active monitoring, shutting down problem IP addresses Let us know when you have IP addresses that aren't working. We now validate that no Nebula addresses are being black holed. we are notified when they are black holed. We will make sure security is communicatig with users Users and PIs will automatically be notified when security issue is reported If other people need to be notified, we can add them to the contact list. Security will try to notify the right people from now on Upgrade to support LDAP environment Easier to create users on Nebula Doug can talk about the identity system at some point You can invite people to get NCSA accounts Account generation is not using paper for NCSA accounts for the past 2 months For Nebula accounts, passwords are still distributed over AFS The new system will do everything over email. identity.ncsa.illinois.edu We want to support Docker outside of CoreOS environment OpenStack wants to provide interoperability with commercial clouds Our goal is for the community to develop and share back with the group Professor Brunner has some research computing going on over the holiday break. He will need to know what days Nebula will have its outage. We plan on less than 2 days of outage Hoping for no downtime Professor Brunner No slides, showed a live demo He has split his research off to separate groups NDAs are required for some of the data. Splitting that data off so that only people working with the data need to sign NDAs He has students from 6 or 7 different departments Set up VM for them, give them Jyperter hub they have IPython notebooks, very interactive collaboration They don't have to wait for a face to face meeting to share things Their notebooks are online and can be looked at anytime Students graduate, move out of groups, need contact to PI Would be nice to have more flexibility creating VMs Sizing flavors Needs algorithms to optimize resource usage Projects cannot create their own flavors - not supported by OpenStack But OpenStack does support as many flavors as we want Give ITS specifications as what flavors you need ITS is trying to keep ephemeral disk as small as possible. Any way to migrate between flavors - haven't tried but should be able to do a resize Can't make it smaller but can choose something with more RAM Snapshot instance first, ITS will test it Roger all flavors have the same root disk - 8 flavors Brian LSST thinks OpenStack uses libvert (?) can go down in memory usage and cpus Restricted can't go to a smaller root disk Roger and Nebula running same version of OpenStack Snapshots use your quotas Can pull snapshots off and put them somewhere else? You can use glance client - not through the web interface Chris create script to download snapshots, Peter to document Has accounting group, creating data analytics course daatascience.business.illinois.edu 40 students in the class, 240 next year Can see the students server, can see what they tried to do Can see when they last accessed their server Allows students to all be on the same environment Teaches info 490, runs jypiter hub server course notes on git hub Fetch assignments, submit automatically Interface allows peer grading Interactive lessons, they get a readonly copy and can refetch lesson Informatics course is completely online Learned - doing data science on the course, know when students are doing the assignments, can see average load How long they stay engaged, can see how students are learning data science TAs and instructors can go into the notebook and see what they are doing February or March will want more servers IDSI - illinois data science initiative Build towards an institute that provides data science across the campus want to leverage resources across disciplines effort to put it together to expose people to data science ideas Key to have something in nebula supporting big data science Don't have capability for arbitrary user to be able to do data science White papers, expertise, how to spin up different environments Ron Yorgasen presented slides