Recently, I've been getting a lot of questions from OpenStack community members around DefCore. So, I thought I would share some of the questions and answers broadly to help encourage the socialization of the plans and progress that has been made over the last year. We have started getting into enough detail that more people can provide constructuve feedback for the entire process.
For those who may not be familiar, OpenStack DefCore is a process being run by the OpenStack Foundation Board of Directors to define how we determine what “OpenStack core” is and how people determine if they can brand their cloud deployment as “OpenStack.” This effort spun up about a year ago and has made enough progress that we are starting to bring the details out into the broader OpenStack community for further discussion and comment. DefCore is often used to describe both the board subcommittee and the process we are following. I will generally use it to refer to the process.
I am a member of the DefCore subcommittee which is co-chaired by Rob Hirschfeld and Josh McKenty. You can find more details about the committee and our processes here:
If you want to know more about the history and how we got here, I would recommend catching up on Rob’s blog posts. He has done an admirable job of capturing and publishing periodic updates about the process over the last year. You can find relevant posts here:
If I haven't lost you yet and you are just looking for a quick update, I'll attempt a brief summary of the DefCore process and some FAQs.
That’s it! Of course, I’m sure that leaves tons of extra questions. I’ll take a stab at a few I anticipate but would encourage you to ask more. I'll work on adding to this FAQ as I get more questions from around the community. Here are a few that I've covered with folks recently.
Q. How are “must-pass” tests selected?
A. Ultimately, must-pass tests will be selected and voted on by the OpenStack Foundation Board. To make the process easier, we have done the following analysis:
We are working on the first iteration of the scoring matrix based on OpenStack Havana. You can review the current draft here: https://docs.google.com/spreadsheet/ccc?key=0Av62KoL8f9kAdFo4V1ZLUFM0OHlrRnFpQUkxSHJ5QWc&usp=drive_web#gid=6
Q. How does the core definition change over time?
A. Our goal is to start with a very small core that is unlikely to have thing removed over the long haul. Then, we will add to that base with each integrated release as capabilities begin to become more commonly used, well documented and strategic to the overall direction of OpenStack. We are using the first pass, based on the Havana release, as almost a “practice test” for this new process. Our release of the Havana based core definition should happen at the ATL summit. We then plan on doing a revision of core based on Icehouse about 90 days after the summit. We want to release a Juno based core definition at the Paris summit. After that, we would expect to update the core definition coincident with each 6 month integrated release.
Q. Tell me more about Refstack. What does the project give us?
A. The Refstack project team is working on tools and website that automates the running of tests and reporting of results. One essential tool is called TCUP. TCUP provides a simple way to run the tempest tests in a Docker container on almost any system. The goal is to make it as simple as possible for DefCore reports to be gathered and reported widely for all kinds of OpenStack deployments. The results of any test run can optionally then be uploaded to a repository on the Refstack site. There will eventially be options for marking results public, managing who can submit official and unofficial results for OpenStack deployments/offerings, etc.
Q. I've seen several references to gathering input from the community. How does that work?
A. The Refstack effort will enable community input in a variety of ways. First, results submitted from TCUP runs will provide a lot of information about how broadly OpenStack capabilities are deployed over time. It will also provide the capability to provide feedback on what capabilities the community ranks highly or may be missing from the initial passes. This will provide a data-driven way for the community to drive further refinement and iteration of the DefCore process. As always, the DefCore meeting minutes are open and our mailing list allow broad participation in the discussions.
Q. How will enforcement work? is there a grace period?
A. While we have not nailed down all the details, we do anticipate giving operators and vendors plenty of time to get in line with the new definitions. Toward that end, our expectation is that nothing will be “enforceable” until at least the Paris summit timeframe. The time between now and then should allow for extensive community comment/revision and for appropriate adjustments to be made. Hopefully, the new core definition can be in effect by the end the year.
Q. How will you know if people are actually running the “designated sections” of code? Is there an audit?
A. At this time, we will be working on the honor system. We expect community members to respect the trust across members and comply with the requirements. If that proves to be insufficient, we probably have larger community issues to address.
Q. Why have the “designated sections”? Isn’t API compatibility sufficient?
A. The board and many community members felt that running substantive sections of the project was a requirement that would require all people using the OpenStack brand to stay involved in the projects. We felt this is essential to maintaining the kind of community that we see as important to the long term stability of the OpenStack project.
Q: Will there be a way for companies to run an API compatible OpenStack? will there be a separate mark?
A: We are not precluding that but it is not the focus of the DefCore project to work on that aspect of branding at this time.
Q: Why do we need any of this? why change things at all?
A: There are two primary drivers. First, we need a concrete answer to “What is OpenStack?” The most important asset of the OpenStack foundation is the brand. If we don’t move to put some specific criteria around the use of that brand, the foundation is at risk of devaluing this most important asset. So, we feel this project is essential to the future of the OpenStack Foundation. Second, we need an ecosystem where OpenStack deployments offer some promise of compatibility and interoperability. While DefCore will not completely solve all of the interoperability issues, it is a necessary and essential step in the process.
Q: What happens to individual projects like Swift, Nova or Heat?
A: Individual projects aren’t directly impacted by these changes. There will still be individual projects that can use the OpenStack brand such as OpenStack Compute, OpenStack Object Storage, OpenStack Networking etc. We are sorting out the exact requirements for determining how a project moves into a position to use the branded name. The goal of this project is to look at the standalone use of OpenStack in regards to broad cloud functionality.
Like many stackers, I was planning to be on a plane bound for Hong Kong in in the morning. Unfortunately, there has been a change in plans. While my shoulder surgery went well and my recovery is going as the doctor expected, it isn't going the way I expected. Perhaps I needed to think it would be a fast bounce-back. After all, it was arthroscopic. Right? How bad can that be. Well, it turns out they can do very sophisticated arthroscoping procedures these days. Two screws, a new hole drilled through my clavicle and two completely severed tendons now (hopefully) getting premanently reattached to my bone is a good thing. It's just not something I can travel on in two weeks time.
So, I'll be here in San Antonio getting two-hour stretches of sleep in my recliner, taking advantage of modern pharmaceuticals and continuing to heal. I'll also be dialing in for the OpenStack Board of Directors meeting on Monday (Sunday night!) I hope the stackers in Hong Kong get a lot done and outline the right priorities for the next six months. I'll be back in April stronger than ever.
The Hong Kong OpenStack summit is fast approaching, assuming all goes well (DISCLAIMER HERE), I'll be there for the OpenStack Board of Directors meeting on 11/4 and the conference the remainder of the week. I'm looking forward to a number of really great sessions. But, most of all, I look forward to the 'hallway track.' Unlike past years, I won't be speaking and I also won't be carrying the responsibility of managing a large organization or running a huge 24x7 operation. This time, I'll be 100% focused on engaging in the summit. With that in mind, I wanted to highlight a few areas I want to dive into during the summit. If any of these resonate with you, look for me at Breakfast with the Board on Wednesday morning or find me somewhere during the week. I'll be the American with his right arm in a sling.
I think this is one of the most important topics that the OpenStack Foundation Board of Directors is leading. Rob Hirschfeld has done a fantastic job of leading the effort to define core. He has pulled in a huge cross-section of the community and put us on a path to move forward on this critical issue. I am very happy with the current direction of having core verified through specific tests and requiring the use of some OpenStack code. It seems as though we have surfaced a lot of the critical issues and are about ready to move forward to the tough work of defining the required details. I am hopeful Rob's goal of having a full process in place in 2014 can be realized. I would love to talk more about this with anyone in the community. Are we on the right track? missing something? totally offbase? Getting input from everyone now will help make sure we keep this on pace to finish.
We didn't move the ball forward quickly enough, IMHO, on this issue in 2013. I am at least partly to blame for that. It didn't receive my constant attention or focus through much of the year. I am convinved, based on the results and community feedback, that we need to move away from the current cumulative voting as prescribed by the bylaws. I have a strong preference to move to Condorcet but Single Transferrable Vote (STV) is another viable option. We finally have some momentum on getting this to the community for vote in the next few months. I would love to get more input on this area. Do you agree we need to move forward on a change? Are we moving too fast? Do you have a strong preference on Condorcet or STV?
The Board and TC made some headway in broadening the kinds of project categories for OpenStack with the creation of Integrated projects. Obviously the board needs to complete work on Core to finish this off. But, the work done by the TC broke the logjam of new projects and some of the confusion around the path for Incubated projects. I would love to get feedback on how people that that is working.
From my own point of view, I dont think we are done with this yet. I trust the TC's decisions around the current new slate of projects. But, over time, I am worried that the overall scope of OpenStack may be getting too broad. The TC and our amazing OpenStack CI team have become extremely efficient at managing new projects. But, these projects all take up mailing list bandwidth, conference time and speaking slots and potentially distract from the focus on core projects.
I don't have a well formed opinion about whether we are in the right place. But, the scope creep and lack of clear definition around what should be managed by the foundation and TC make me a bit nervous. Do you think we are on the right track? Should we be expanding faster? or slower? Am I worried about nothing? Am I just too old and conservative in my thinking? I would love to engage with people on this topic and get a better sense about other feelings on this issue.
As I pointed out before the Portland Summit, Rackspace needed to reenergize our efforts in the community and close some alignment gaps with common OpenStack practice. We've largely stayed on track with these efforts throughout the year (I intend to have a detailed update on the Rackspace blog before the summit). But, I know we can do more. I think we can make improvements in our Rackspace structure and processes that will enable us to more consistently engage in the community and keep our offerings more aligned with common OpenStack practices. If you have experience with organizational models that allow product/service companies to effective engage in upstream while maintaining customer facing obligaitons, I would love to hear more. If you see places where we are completely missing the mark, let me know. I'd even listen to where we are getting right! Let's share ideas and make everyone better.
Assuming I can get used to not using the "Q word", I want to dive deep into Neutron. Our early efforts around the networking projects yielded a product last year. However, the tradeoff was that we've diverged from the current code and we haven't been involved in upstream development. This is especially true for myself. The completion of some stability projects along with some new people on the team give us an opportunity to reengage. I would love to talk about how we help get pluggable IPAM so that Rackspace can move towards an upstream networking implementation. I would also like to know where I and other Rackers can begin to help and get more tightly integrated across the Neutron project. I want to see the transition from Nova Network to Neutron go smoothly. I think it is important for Neutron to become the OpenStack differentiator it can be. I'll be in a number of Neutron sessions but feel free to grab me for some deep discussion around all of this.