Precis
This is a bit of a story, a bit of a history, a bit of a documentary, and hopefully a lot of entertainment and theatre about Shadowcat Systems.
We are ten years old and to celebrate that fact I want to write ten unique little pieces about Shadowcat that hold as a memory of our first ten years and a celebration of that decade down. All of these pieces are scheduled to be released throughout the whole of our 10th year
Commerciality
It might seem a little strange to be talking about commercial products when we are primarily a consultancy. There is an easy answer in that when you are a consultancy the services you offer are your product, and although that’s the case it isn’t always a true analogy.
I am going to use the term to refer to specific products or services that we have created. When I say created I don’t mean for clients or partners as we have done many of those, I mean ones that were, and are, internal to Shadowcat. This isn’t a comprehensive list but I am going to try and cover the main group.
Commerce Builder
This is probably the reason that Shadowcat is in existence, if we disregard the fact that the principal catalyst was pTerry Pratchett (see without pTerry). Back in the early part of this millennia, at the dawn of a new century, Matt and I built a website management system and commerce system in Perl. We called it Commerce Builder.
The original Builder was created for the publishing company that I worked at, it was a principal component in the evolution of Shadowcat, and was then sold to other people who wanted similar functionality. There were a few advocates of Commerce Builder as these were the days of fledgling website CMS. Written in Template Toolkit, and I believe HTML Mason with a lot of homespun code from Matt, it was a big learning experience. We learned that what seems to be a simple task of creating template systems to handle automation is in fact a cul-de-sac leading to a labyrinth of individual customisation. We learned that the most problematic customers are those who pay the least and want the most number of just ‘small’ changes. We also learned that we should have stuck with Commerce Builder and refactored it in Catalyst, Dbic and modern front framework such as Bootstrap or Angular.[1]
CB had a steep learning curve in the generation of pages and automating of elements in the framework. However it was also very powerful. A single install was able to run multiple sites in a manner that more modern frameworks now emulate, and they have hit all the same issues we did. We eventually retired CB when other more popular frameworks came around and as I mention in the notes I sometimes wish that we never had.
ShadowNMS
ShadowNMS is a network monitoring product that we have had in the pipeline of development and available to select partners for the last year. We have made a move to automate service monitoring and discovery using a number of tools. The Live Beta is already available and we do have payment options. Though it is still under a lot of development in the background.
Some of the screenshots for ShadowNMS.
A lot of the development is focussed on evaluating and testing different frameworks, APIs, plugins and systems to make ShadowNMS a great tool for a variety of businesses. The issue with monitoring is that everyone needs it, but few people know either how to do it, how much they need it or how to do it well. Also it is expensive, in both knowledge to set it up and systems needed to run. So there is a lot of overhead which means that the only people who do monitoring to any great degree are very large organisations or dedicated enthusiasts. The rest of the Internet relies on someone up the line handling the issues for them.[3]
Hopefully we can hit a stable ground between limited service options, aiming to satisfy the most common problems and concerns, and cost. In order to broaden the field and at the same time lower the barrier to entry we are assessing a multiple of approaches. Our aim is to start with some smaller ground of discovery, information and service and then to add features that are required by customers and partners, growing the service for those who use it. We are also looking for product partners so if you are interested you should give me a shout as we can give access to the system for trials and discuss options.
Timebot
Timebot is an internal tool that we use for monitoring times and projects across a multiple of developers, time zones and locations. It works from a command line via an irc bot, or from a gui directly to a calendar. Since it uses some open APIs we have also had developers who automated it to Vi and Emacs allowing them to log directly from the VM.
Timebot is now in a 1.5-2.0 transition period where we are adding on some visual reporting and also working on a browser based input and recording system. At this point there is no plan for a commercial version but I mentioned it here as it is a fairly robust product that we have under active development and may one day work its way out of the confines of the Castle.[4]
Support Contracts
Commercial Support contracts are normally the domain of proprietary languages, technologies or designs. Even in the open source world there is a measure of restriction to their adoption with products such as Magento having a commercial ‘extra powered’ version and an open source version with less support and older feature set. Far more popular are support contracts of 'expert help'.
We like to offer Support Contracts aimed at our special set of skills. Shadowcat are fairly expert in setting up strong environments, systems automation, code refactoring, integration planning, debugging and architectural planning. Our developers have worked in a number of diverse environments and are familiar with the common pitfalls and issues as well as adept at finding the issues unique to each business.
Our support contracts are aimed at not just providing a safety net, or insurance, with a pot of money that is an offset against issues and lost each period. However for years we have struggled with the issue that if the hours are not used the money 'feels like' it has been wasted. It hasn't, of course, I am not suggesting that people try to fleece companies with support, they don't and I know there is a lot of value in having the expert support, peace of mind and quality partnership. It has just irked that we have a pot of funds that if not used 'could have' been used.
We like to look at supplying our support partners with a service where we use any contractual sum to mitigate issues. This may be by auditing environments, automating systems, regular review or planning, system duplication for testing or on building a good workflow for distributed development. We like to think of the Support Contract as a block of hours that either get used in expert assistance, development, automation or training and support. We also like to work with them to use it to support the community by improving the modules and systems they rely on. It is, essentially, preventative maintenance and system robustness, by pushing back changes we support the whole eco-system.
No two clients are the same, no group of developers have one mindset so we customise our support to provide the best value service for the client. We also help to build a stronger community in the same manner. By encouraging our clients to return patches, changes or evolutions of software back into the community we hope to make a better environment for everyone.
Closing Words
There have been other products and services that we have developed, trialed, abused and retired over the last ten years. These were just the few that seemed most fun to talk about. I miss the opportunities we had with some of them and wonder if we should re-imagine them for the current times. I personally always liked PicBot but it never went very far. The Paste bots and other small irc bots are still in use and we have a number of system admin tools that are part way towards our grand project.[7]
If you want to know more about the above just drop me a line or use the comments, thanks for reading.
Previous Articles
[Don't forget that you can join in this conversation by using the comments form or by tweeting at @shadowcat_mdk]
Notes
[1] Unfortunately this last lesson we only learned about three years after we stopped developing Commerce Builder. I can probably daydream about going back and remodelling it for the modern web, but I suspect that Matt and I could not get drunk enough for that to ever happen...[2]
[2] [/me looks at the code] Dear Gods my eyes, my poor mind is leaking...
[3] There are a lot of issues in this area. It is true that you should expect a level ofg service assurance from the providers you pay upstream. So you expect them to monitor connections and hardware etcetera. However that doesn't really place an obligation on them to monitor all your services unless you specifically pay for that. Also there is value in making sure your services are performing well and are constantly tuned and evaluated. How do you do that without some level of monitoring? Then there is this question: Do you want someone else to tell you that your essential services are unavailable, or do you want to know either first, or at the same time they know? For the micro business website availability, email availability and basic connection status is probably more essential than it is for larger, more robust, organisations.
[4] One of the strengths of the first Timebot was that it used the Google API to the Google Calendar allowing someone else tyo do some of the grunt of 'available on all platforms' visual status. These days we are moving towards a more fluid and autonomous internal 'bot.[5]
[5] There are a number of anecdotes about TB, especially when it has a mini-breakdown and gives some exceptional or unexpected behaviour. Recently it has been given textual abuse and shown the hopper.jpg that it can go 'suck on'.[6]
[6] I piqued your interest but just don't ask to see. Once seen it can never be unseen.
[7] Grand project you say my dear, what is that then? Well you will have to wait, it isn't a secret... but it has its own thunder. It is about automation, management and installation of software. It's a tough nut that our brightest brains have been smacking up against for over a decade.
One Decade Down
Part Seven - Open Source Software Consultancy
Precis
This is a bit of a story, a bit of a history, a bit of a documentary, and hopefully a lot of entertainment and theatre about Shadowcat Systems.
We are ten years old and to celebrate that fact I want to write ten unique little pieces about Shadowcat that hold as a memory of our first ten years and a celebration of that decade down. All of these pieces are scheduled to be released throughout the whole of our 10th year.
An Open Source Consultancy?
We have always prided ourselves on being a good Open Source Consultancy, but what does that actually mean to us?
It might mean that we primarily use Open Source, which we do, in order to reduce costs to our clients. It might mean that we use Open Source Tools as they represent the best of breed in a modern world, as proprietary software is often seen as sluggish, de-innovative and filled with protections that prevent adaptability and evolution. It might mean that we engage with the philosophical concept enshrined by libre, that free software allows the sharing of not just work, but ideas, concepts, that it inspires instead of restricts. It might be that we love communities and contributors.
The truth is that it is of course all of these. One element we pride ourselves on the most is that we are upstanding members of the community. Being part of Open Source isn’t just using the tools or agreeing with the methodology, it is contributing to the projects.
Over the years Shadowcat has created, or contributed, to a large number of projects, but some of the most memorable are:
- Dbic (DBIx::Class) (http://www.dbix-class.org/) (https://metacpan.org/pod/DBIx::Class)
- Moo (https://metacpan.org/pod/Moo)
- Local::Lib (https://metacpan.org/pod/local::lib)
- Fat::Packer (https://metacpan.org/pod/App::FatPacker)
- Devel Declare (https://metacpan.org/pod/Devel::Declare)
- Reaction (https://metacpan.org/pod/Reaction::Manual)
- TAK (https://metacpan.org/pod/Tak)
- Object Remote (https://metacpan.org/pod/Object::Remote)
- Web Simple (https://metacpan.org/pod/Web::Simple)
- Catalyst (http://www.catalystframework.org/) (https://metacpan.org/pod/distribution/Catalyst-Runtime/script/catalyst.pl)
- Dancer (http://perldancer.org/) (https://metacpan.org/pod/distribution/Dancer/bin/dancer)
- Moose (http://moose.iinteractive.com/en/) (https://metacpan.org/pod/Moose)
… there are many more
You Possibly Used our Software Already!
So this is an anecdote that I like to use, mostly to non-technical people, as it shows the influence of developing and engaging with community software. The conversation goes something like this:
- Me: So you have probably already used some of the software we originally developed.
- Them: I doubt it.
- Me: Have you ever used the iPlayer?
- Them: Yes.
- Me: In the UK?
- Them: Yes.
- Me: So we created a piece of software called DBIx::Class and that was used by the iPlayer team at the BBC to power some of calls to databases on servers.
- Them: Really?
- Me: Yes, so there is a good chance that you used some software that we created and contributed for free.
- Them: Wow.
Or sometimes…
- Them: I bet you wish you had charged for it now?
Which is really missing the point. However to my knowledge there are still parts of the iPlayer substructure that use Perl and more specifically Dbic as an ORM, so even today there is a good chance that a large number of people have used some software that Shadowcat (more specifically Matt S. Trout) originally created. The project has now over 200 contributors and is a standard module for many people working with Perl and databases.
That’s one of the beautiful elements of working with Open Source and being an OS Consultancy, you never know what big thing you are going to be a part of and a whole lot of people are willing to help you create that just because it seems like a good idea to them as well.
Previous Articles
+10 Years Young +The Alumni +The Extended Family +Interns +Clients +Commercial Products
