DevOps for Content - Is this for You?

 

The standard definition of DevOps is a set of practices that automates the processes between Development and Operations teams for a quality product to be built, tested and released quickly and reliably. DevOps is always thought of in the context of software development, we, however, decided to apply its principles to content development and would like to share the results of what we learned with the audience today. 

What content were we producing? 

Framing of this problem statement started with 60 hours of technical content that was to be produced within a 6 month timeframe. Add accessibility, accuracy, videos and other rich interactions as one has normally come to expect with high fidelity content these days. Despite having a strong delivery team of Subject Matter Experts (SMEs), Instructional Designers, Media Engineers, Quality Control Personnel and Project Coordinators, the team felt that they were unable to meet product management demands on releasing content modularly with the quality that our audience has come to expect. The result was that the content was delivered on time, but having to release the course in its entirety, without the ability to release it in chunks for faster market feedback. This left the development team with the following questions:

• Can we produce content in smaller chunks? Modularly? 

• Can we release it faster?

• Can we release it as it is built? 

• How much can we automate without impacting pedagogy? 

The team decided to try a different approach for their next course, which also happened to be their flagship product to address the questions raised above. 

The DevOps Approach 

Amongst many others, DevOps essentially advocates the following principles which our team found very useful:

• Version control all artifacts 

• Continuous integration and deployment to various environments 

• Automate everything you can 

• Learn from Failures 

• One Team, One Target 

Earlier this year, the team decided to meet in person and orchestrated a DevOps Workshop, where product management outlined what they were looking for in terms of release cycles for content and the expected volume. The team then devised a two pronged approach to bring the DevOps principles outlined above to life.

Process 

The team already was following agile principles and practices to build content, so it was relatively easy to shift their thinking to “ToDo Blocks”, “In Progress Blocks”, “Review Blocks” and “Acceptance Testing Blocks”. The goal was to minimize In Progress blocks and move as quickly and efficiently as possible to Review Blocks. The team also developed limits to how many blocks were in the “In Progress” and “Review” states, to get finalized content as quickly as possible. The team was organized initially into separate Content creation, Media Engineers and Quality Control teams, and with the shift to DevOps, the team moved to having a single Sprint demo and joint planning sessions to facilitate greater interaction and hand off. 

Desiging Content Modularly 

Early on, the team coined the term “blocks” which allowed for content chunking at the enabling learning objective level. This concept then allowed the tooling to be developed at this level with the appropriate competency mapping for each asset at that level. 

 

Tooling 

Separating Content from the Interface

A common and generally recommended approach to web development is to separate the user interface (UI) layer from the content layer. Most often, the UI is making API requests to a server, receives data, and renders the new data on-screen. This allows for separation of concerns in that Front End developers can focus on the UI layer without being concerned about what the back end is doing. In our case, our UI layer is React, written with Facebook’s Create React App developer tools.

Content as Code

The product of our work is standalone sites that don’t require a database or backend processing for content delivery. Content is served as static files files to the Interface, and the interface processes these static files accordingly. When considering what file and data formats to use, JSON is an obvious choice. However, when considering the DevOps model, where this data will be worked on by content authors of varying skill levels, JSON becomes a bit too strict, and a bit difficult to read. Since the majority of content authors are used to working in MS Word, working directly in JSON would become a cumbersome task. On the other hand, YAML (which converts to JSON efficiently) is a much easier syntax to read and work with. Even with YAML, however, formatting actual display content still becomes tedious, and that’s where Markdown comes in. Our current data consist of a mix of YAML for providing structure to various components, and Markdown for the content. Markdown is a great bridge for Authors used to writing in MS Word, as there is a plethora of Markdown writing software out there. Shout out to Typora.

Word Transformation

Even with easy-to-read languages like YAML and Markdown, writing it all out is still time-consuming and tedious. Our data structure itself leans heavily into the concept of modularity (blocks), and our text content is broken apart into individual assets. This makes writing and entire page fairly complex. With that, in order to enable the rapid production of content, while adhering to a modular approach,  we’ve written scripts for the authors which take a formatted Word document, and parses them out in to their appropriate YAML and Markdown formats for consumption by the UI. This has provided authors with a stop-gap CMS, if you will. It gets their content as code about 80% of the way there, while keeping the familiarity of MS Word and the review processes the authors maintain when developing their content.

Version Control

Since all of our content is static files, it’s perfect for any version control system. In our case, we’ve got with the current industry standard for a VCS: Git. Using Git, we’re able to track all of our content, revert to previous commits, branch off and explore different concepts without having to worry about affecting the main content. Git makes breaking changes nearly impossible to undo, so restoring to a previous working commit a snap. No content is ever truly lost within Git.

Product in Context

Without a CMS, working with content on a Content Author’s local machine is a bit of a challenge when they aren’t able to see the results of their work. Our technical requirements for local setup are pretty slim: Node and Git. And with some setup instructions and repository permissions, authors have been set up with the same local environments as the developers. This has made viewing content changes in the context of the actual product itself easy, resulting in some serious time saved. The authors can adjust content, get immediate feedback, and know for sure what their content will look like once deployed.

Delivery Pipelines

Working within Amazon’s AWS suite of developer productivity services, we’ve established build and delivery pipelines to ensure our work is visible to stakeholders as close to immediately as possible. Empowered with local environments and parsing tools, Authors are able to create content in Word, transform the content to YAML and Markdown, verify the changes, commit and push to our Git server. From there, AWS CodeBuilds kick off, performing additional content build tasks before deploying the content to an Amazon S3 bucket with CloudFront as the gateway. From content push to visible changes, the process is about 4 minutes.

Communication

Communication might be the most valuable tool in your toolset. The DevOps concept of “Failing Quickly” is that you work closely together, try something out, and keep trying until you get the intended outcome. You can’t fail quickly if you’re not communicating often. With that, our team is armed with several Chat channels, and meet often to discuss progress, blockers, and solutions. 

Presentation and Demo 

As you walk through the demonstration area, we hope to show you a small glimpse of the course, a module that was generated through the DevOps process and a view into the tools described above.

In Conclusion

The team was able to produce an Alpha module of a bigger deliverable that is due November 2019, using the approaches described above and has already seen improvements not only with collaboration but how seamless the tooling and processes were. We also learned that finalizing the workflow and the Content management system approach could have occured earlier in the cycle, but considering timelines and priorities, the Word-tool approach the team adopted actually allowed them to have a stop gap, highlighting areas that they are now working to address, which also validated the “Fail Quickly” principle. 

As we continue to hone this process, we hope the DevOps Mantra resonates not only with our teams but with yours too. 

 

Feedback and Questions after the demo: Sticky Notes to collect responses 

• In working with content authors, what challenges have there been surrounding adoption of new tools and processes?

• In working with Operations, what are some challenges that have resulted in longer project timelines?

• In working with a development team, how did you solve challenges that arose?