In writing my recent post on what I now think about Continuous Everything, I realized that there was some missing context - my own personal journey of 20+ years down the rabbit hole of the revolution that’s happening in the practice of software delivery.
It’s a journey from fearful resistance, to limited adaptation, to radicalization.
Looking back today, it now feels like destiny to be here as part of a team that is systematically rethinking every function and process in our own business through a continuous lens -- not just for our own benefit, but because we are building the software system to power every other business in the continuous economy. Yet along the way that wasn’t so clear. If you’re curious about how one person can transform their thinking radically, read on.
First, a few quotes, to set the tone...
“My dear, here we must run as fast as we can, just to stay in place. And if you wish to go anywhere you must run twice as fast as that.”
― Lewis Carroll, Alice in Wonderland
“software delivery is an exercise in continuous improvement, and our research shows that year over year the best keep getting better, and those who fail to improve fall further and further behind.”
― Nicole Forsgren, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
“But I don’t want to go among mad people," Alice remarked.
"Oh, you can’t help that," said the Cat: "we’re all mad here. I’m mad. You’re mad."
"How do you know I’m mad?" said Alice.
"You must be," said the Cat, "or you wouldn’t have come here.”
― Lewis Carroll, Alice in Wonderland
“much of what has been implemented is faux Agile—people following some of the common practices while failing to address wider organizational culture and processes.”
― Nicole Forsgren, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
Milestones on my journey
1997/8. I’m a 23 YO fresh hire at Microsoft in the newly formed Microsoft Online Operations group. I’d already been working for 5 years in SF Bay Area startups in various business operations and field engineering roles -- so the folks in Redmond thought I knew how digital and online worked (never mind my prior companies were in multimedia tools and telco/enterprise class software that released features every few months and took 12-18 months to get a deal closed.) I get new assignments to sort out parts of our online ops mess every week. This time I get sent to Bothell, WA to meet with the guru of Microsoft release process for shipping software on CDs printed at CD controllering houses and distributed through retailers like Egghead. I’m told to work with her to adapt their months-long incredibly convoluted process to what we need to do for online releases. She comes in and drops a 3-inch thick binder of pages of checklists on the table. My instinct is that this is not simply an edit.
1999. I leave Microsoft after the IPO of my pre-Microsoft company, Portal, because I have the means to seed my own thing. I start an art dot-com company, Artloop. I raise a $3.7 million Series A. I hire PMs and content people but I contract out development because hiring engineers is almost impossible in San Francisco at the height of dot-com hysteria. The big bang launch of the site is delayed and delayed as the development site looks and performs like $*!&%&#*@. I can’t afford to quietly let the development site go live and iterate because of the contract terms with my development outsource company. By the time it launches the B2C crash has happened and I’m desperately trying to sell a B2B pivot to raise Series B to keep the lights on. Ironically, the actual product idea is about creating a real-time, crowdsourced, organically and iteratively growing database of rich art knowledge that can start small and asymptotically approach quality and comprehensiveness. Did I connect the folly of my delivery approach to the site to the deep truths of the approach to content? Of course not. Oh well, by the end of 2001 I was broke again and out of a job so ripe to go make some new mistakes.
2003. I’m a newly minted 29 YO VP Product Management at a venture backed enterprise log management software company where I was previously VP Professional Services. I’m excited to be in charge of THE ROADMAP! My board is a bunch of sales types who want to know when on the roadmap our log management system intended to report on user behaviors and performance metrics and find error messages will completely pivot to become a SIEM because they heard ArcSight’s sales were taking off. My enterprise customers including Bank One and Lehman Brothers (notice a pattern of spectacularly demised or silently absorbed brand names so far?) demand roadmaps exactly when they will get all of all the features our salespeople promised them. And then my out of control engineers who hate PM and management discover this “agile” thing? NOOOOOOO!!!!! We’re going to commit to that pivot the board wants and it’s going to happen in 9 months, I PROMISE!
2005. That pivot didn’t work out so well. Our 4.0 complete frontend rebuild and brand new real time correlation engine is a year late and counting. We haven’t delivered anything meaningful to customers for longer than we can remember. All of this agile sprinting is resulting in one impossible to love or sell internal preview after another Friday demo after Friday demo. When the build doesn’t fail.
I jump ship in March of that year and become the Chief Product Splunker at the then-12 person Series A Splunk that’s still in stealth mode. There’s a main branch that hasn’t been able to compile for weeks and at least 12 competing ideas of what features need to be in a viable beta. The 9 prima donna engineers here heard about that $&@!*&@!#(@! agile thing too. Damn! No matter. I’m going to write a 60 page 1.0 MRD and build a roadmap and get them to comply. They laughed at the roadmap but we did manage to get a somewhat viable 1.0 beta released for public download with some coherence in about 4 months from that point.
Interestingly, purely in hindsight, I realize the reason users start loving Splunk itself is because they’re either sysadmins supporting or devs developing for various emerging kinds of web sites and services. Services that are being continually changed and break in new and interesting ways with all kinds of mysterious errors to debug against increasingly fragmented service oriented and web 2.0 architectures. Our customers, like me, were coping with the emerging continuous model.
2006. We ship 1.1, 1.2, 2.0, seemingly a release every 6-8 weeks. We have lots of free users and some paying customers. A growing stream of feedback is coming as support cases, forum posts, in person sessions helping customers install and use the thing. I’ve got the more mundane title of VP Product Management & Support since we have real enterprise customers and closed series B funding with a more conservative firm. I have support engineers but none of my PMs last. Engineering has made sport of alternately mocking and ignoring my roadmaps and MRDs, but they listen when I bring them one issue at a time with real users and their stories and requests attached. Hmm. Can I automate managing these discrete issue backlogs? Let’s see what’s available.
Engineering is using Jira running on Linux servers in our server room to track their backlog and still talking agile. But it doesn’t know anything about PM process. I look into some PM requirements and roadmap tracking tools of the day. They all run on Windows. They have some good ideas around automating the bare bones of the well known Pragmatic Marketing “requirements that work ” framework to track enhancement requests, other market datapoints, customer call reports and massage them into market problems.
Our grumpiest cofounder and chief architect is also our IT guy. He tells me no way no how is he running a frickin’ Windows box. Why can’t you use this Jira thing we’ve already got running? (Long before Jira as a SaaS.)
So I start learning about how to customize Jira to do my PM stuff too. I think about how to continually feed support tickets captured via emails and support portal case submissions (all in Salesforce.com’s support module) into inputs to a product management process in Jira. I document some requirements and hire a consulting firm to customize Jira how I want it.
Separately, I also own documentation. All of these minor releases require that every existing topic be examined for how it should change. Support is constantly figuring out new solutions to problems that we didn’t anticipate or design for as cases arise. Engineers are starting to blog technical information on their personal company blog accounts that they don’t bother to go back and revise when the software changes. Updating the full manuals through the Robodocs type tools of the time is impossible. Knowledge bases would end up being a set of deltas to manuals bigger than the manuals themselves! Everyone tells me just to make the docs a wiki. That isn’t a panacea because we have customers that don’t immediately upgrade and need to find docs from a few minor or even major releases back. How can I have formal per software version manuals with structure that are updated in real time by many participants like a wiki?
OK. I go talk to Ashley who runs our website about how to do that. We start defining and he starts building a custom docs platform that can be edited in markup live like a wiki but where we can have docs for different software versions live simultaneously. We start using it and trying to get people across the company to edit live docs when they make a change or notice a problem. A few do. It at least keeps our docs writers a bit less crazy.
Besides docs and PM requirements process, there’s one more major thread that came to my awareness this year that’s relevant to the continuous revolution in software delivery - our self service, freemium distribution model. Unlike the pure top down enterprise sales model companies I’d been in before, customers have to figure out how to use our software - and stay engaged long enough to do that - all on their own without a sales engineer holding their hand. If they run into roadblocks we don’t see them directly. We’ve put some basic instrumentation into our UI with a banner in the on-prem product that’s served off of splunk.com, with the version and license level - free, trial, or paid - baked into the URL request. This, in combination with a cookie gives us a basic view of how many downloads turn into installs and successful first launches of the UI, how many users keep using, how many users are on what versions, how the conversion seems to be tracking by version. We don’t have the visibility of a modern day SaaS, but our little banner gives us some nice data.
This behavioral data supports an iterative process where we make tweaks to the first run and in product help experiences every single minor release to see what converts best. Should we load a sample dataset and make that the default search? Should we pop up a wizard to guide loading a first dataset? How about a Splunk assistant? Make the Splunk assistant Smurfette for fun? (Yes, we had our very own spawn-of-Clippy and it assumed a certain blue lady’s identity in our short lived optional Smurf UI theme.)
Despite all these emerging continuous practices, I still accepted the pressure on a young PM exec to produce roadmaps. I’d manually publish roadmaps that were pure fantasy while trying to build reports off of Jira of upcoming features that update automatically as engineering iterates that I would call roadmaps. They’d change all the time, but at least I’d have something to show the board and the big companies we hope will buy us bigger deals than self-service was getting us. Never mind that engineering keeps laughing at me and I keep having to explain slips and changes to the board and customers.
2008. I’ve got my systems dialed in. I can deal with these agile fanatics now and still do a professional job of product management and documentation!
My product management Jira project is automatically getting enhancement requests from Salesforce support tickets. Our PMs have workflows in Jira to scope those to problems for engineering to work on. They define features there too to establish the names and positioning of what’s in releases as the solutions to the problems come into focus. All of this updates back to Jira so customers and support can see what’s happening.
Proudly, I start talking at conferences aboutdoing product management for agile - but my tone is still one of reluctantly accommodating it rather than embracing it. I don’t realize that we actually have the bare bones of both Continuous Product Management and Continuous Product Marketing. If only I wasn’t so stuck on those roadmaps. Which, predictably, keep earning me laughter from engineering and uncomfortable board and customer meetings.
Ashley and his team have replaced the first gen wiki docs platform with one based on mediawiki that eventually became the later open sourcedPonydocs which has magical abilities to manage topics across software versions and support crowdsourced real time updates.Rachel Perkins runs docs and the culture of everyone in the company contributing technical howtos, fixing errors in docs, and Rachel and her team watching changelogs on the docs and in our Perforce SCM to make sure things don’t slip through the cracks is firmly established. Customers with splunk.com accounts can comment on any topics and we watch that too. Our new VP Support,Lionel Hartmann initially wants to have a knowledge base but I successfully resist - why do you need a knowledge base to patch wiki based docs that you can fix directly? - and eventually he gains the fervor of the converted. I don’t know yet to call it that, but effectively what we’ve got is Continuous Documentation.
2009. We have a huge 3.x Splunk user base in the millions and paying customer account base in the thousands. But they’re getting restless because we haven’t delivered much lately. Again I’m a beleaguered VP Product in the middle of a big 4.0 that’s a year plus in the works with no sign of completion and a restless customer base. It’s feeling like 2005 all over again. And I’ve got a new CEO that showed up late the previous year who’s breathing down my neck about 4.0’s scope, design and expected delivery date. It’s a massive release that has to attract new customers and users, and allow all existing deployments to upgrade and migrate. We have a beta process, but no way to get production feedback on iterations of the new release before it’s everything to everyone. Eventually we agree to release a 4.0 that is scoped down to new and existing enterprise license customers and we abandon the free customers for a while, leaving them on 3.x, without alignment on a plan to serve them in the future. Scoping down and iterating - kind of the right idea. Not communicating and abiding by promises to our larger community and being honest we have to iterate - not so much. And the first iteration was still too big and high stakes and took too long to get real feedback. I’m still absorbing the lessons of that year and the following.
2011. I’m now in charge of both PM and engineering for our new commercial apps product line that builds on Splunk core as a platform. I’ve got commercial solutions in development for enterprise security to make us a contender as a SIEM (everything old is new - see 2003!), and for VMware management. I’ve got a new distributed engineering team between Taiwan, San Jose, San Francisco & Chicago when Splunk core which we depend on has a close-knit San Francisco in-office engineering hotshot culture. We’re under pressure to commit to a 1.0 GA date and 1st year revenues for the new products. Any bugs, performance issues, customer feedback on our initial releases is considered proof the projects are ill-conceived and our new team is incompetent. We have no space to experiment, fail, learn, try again. Blame is everywhere.
We only start to get traction once we stop trying to get everyone to sell our new products to every Splunk customer - many of whom by this point are hardly early adopter types - and we get a handful of sales people and SAs who see the potential and understand software delivery - to sign customers who know that we are iterating fast on something new. Oh, and I hired a VP Engineering for that team that knew how to explain software delivery realities to the business and help engineering teams feel incremental success. He and I still feel the need to delineate major/minor releases and plan them on a roadmap, but we at least start delivering value to customers. (Incidentally security driven by this enterprise security suite drives high double digit percent of Splunk’s multi-billion dollar annual revenues now - check their public filings.)
In hindsight I almost completely failed because I didn’t challenge big, waterfall, big bang release assumptions, yet again.
2012. I left Splunk to go to Zuora as its SVP Product. I thought I wanted to do something new and Zuora appealed because subscription billing seemed to pick up on the billing thread I abandoned in 1997 when I left Portal Software for Microsoft. Company and leader with justifiably famous vision. While I left after 6 months because I realized post Splunk IPO I could and should really take some time to myself, the ideas Zuora was developing about the subscription economy are clearly the business dimension of the continuous economy and continue to be an inspiration.
And at the same time, this was my first experience leading product for SaaS - so my firedrills revolved around changes in live systems affecting all customers that had unanticipated problems; and challenges making changes for specific customer implementations on a multi-tenant platform. Some very important learnings to file away for later reflection.
2014. I start a new art tech venture, Aura. Artloop lives! This time it’s mobile vs web centric and we realize that the tens of millions of people in the world taking pictures of art on museum, gallery and art visits with their phones is a great raw source of the data. More continuous learnings here - this time about what a pain mobile development is because of the poor frameworks to get code changes into the hands of a mobile app private test group, and the impossibility of dealing with mobile app marketplace lead times. We don’t succeed for other reasons but it was good to get some hands on learning in mobile to go with prior SaaS, enterprise software, freemium software and web experiences.
2016. I land as CPO of Interana, the behavioral analytics company with a Facebook Scuba-inspired exploratory interface and a columnar back end with special capabilities to look at and segment on behavioral sequences. One of it’s biggest customers is Microsoft Bing, who uses Interana to observe whether behavioral changes in production for their 3x/day releases match product manager hypotheses, or whether there are unintended negative impacts. It’s test-in-prod and it’s mission-critical for enabling modern continuous delivery. Tons of learnings from them and other Interana customers. Meanwhile for my own PM team responsibilities I start trying to turn around some of the usual negativity about roadmaps by making a big deal that “paper real world roadmaps don’t have dates, so why is that the assumption about a software roadmap?” I’m fighting a losing battle. At the same time, over in engineering our leadership makes real strides in getting more regular and more delivery of innovation by adopting continuous integration and eventually continuous delivery - and I start learning a bit about the development tooling that is making this new world possible.
2018. CloudBees comes calling. And the more I learn about its central role in making CI/CD a reality for the world’s largest and most innovative companies, and start connecting that to my own experience with aligning related areas of PM, docs, marketing, support, monitoring and behavioral analytics to agile and continuous, the more I realize this is where I’m meant to be.
As Sacha says, Onward!