Intro Into QuiNovas Open Source

QuiNovas has published over 130 open-source projects to date, and we continue to add many more. Predominantly these projects revolve around our work, focused on DevOps in AWS and Serverless architectures in AWS.

In addition, we contribute to serveral open-source projects, with major contributions to date in python-hl7 and pydicom.

All of our open-source projects may be found in our QuiNovas Github.

Building all these open-source projects has been very rewarding (and very useful to our consulting business), if no one else knows about them they they are the equivalent of one hand clapping. For our efforts to really be useful, we have to get the word out.

In this post I will be highlighting some of the various projects that we have built. In subsequent posts in this series, we will be taking a deep dive into selected projects that we feel are useful to the general community.

Project categories

Terraform modules

QuiNovas began life focusing on filling the DevOps gap that exists in many(most?) companies, and we do much of that work using Terraform.

Terraform modules are reusable DevOps components that are published to Terraform’s module registry. Some highlights of our modules are below:

• terraform-aws-lambda - This module creates a Lambda function in AWS, including the required role, log group, and invocation policy (for use in other roles). Used by us and others over 308,000 times, and our most popular module.
• terraform-aws-dynamo-table - This creates a DynamoDB table, Global Secondary Indexes, and autoscales them to specification. Used for generic table creation (for single table creation, use terraform-aws-ddb-single-table instead) over 71,000 times. We are very heavy users of DynamoDB!
• terraform-aws-appsync-lambda-datasource - This module creates an AppSync datasource for a Lambda function. We do a lot of work with AWS AppSync, and most of our AppSync API implementations are sourced by Lambda functions. Used over 15,000 times.
• terraform-aws-cloudfront - This module creates a cloudfront distribution backed by an S3 origin bucket that it also creates. We believe heavily in static sites, which means we set up a lot of CloudFront distributions (you are reading this from one). Apparently a lot of other people also like CloudFront - this module has been used over 12,000 times.
• terraform-aws-route53-public-zone - This module creates a Route53 public zone. Managing AWS Route53 (or any DNS) can be kind of a PITA, even using a tool as powerful as Terraform; this module makes that easier. Used over 12,000 times.

We have published over 50 different modules designed to make our lives easier as we build infrastructure-as-code architectures for our customers. They may be found here, in addition to being hosted in our Github repository. Check them out if you build stuff in AWS.

Python projects

We used to write server-side software in Java, Node, C#, PHP (yes, I know, he’s our black sheep). But then we found Python, and now almost everything that we write is in Python.

Most of our Python projects are focused on our work in AWS. We use all of them, and sometimes use them daily. They are battle-hardened. A few of our favorites are highlighted below:

• lambda-setuptools - This was our very first open-source project. Standard Python builds use setuptools to build source distributions and binary distributions. However, AWS Lambda requires a different distribution format than either of those. lambda-setuptools enables AWS Lambda package builds in the standard build system by plugging into that system and offering additional targets. You can build lambda functions and layers, upload them into AWS S3, and deploy them to existing AWS Lambda functions.
• cognitoinator - We built this project to make it easier to have AWS boto3 applications that log onto AWS Cognito and then assume a role on AWS for resource access. In addition to simplifying your application’s access, this packages maintains refreshed session tokens for you, just like the rest of boto3 does. The name is a shout out to one of my favorite cartoons, Phineas and Ferb.
• s3-bucket-replicator - Sometimes you just want replicas - lots of them. This little project is an AWS Lambda function that watches a primary S3 bucket and replicates changes to as many S3 buckets as you like. The built package is hosted in our lambdalambdalambda-repo repository buckets. These are public, and you can access them by appending your region name (with a leading “-") if not in us-east-1 (the default). In addition to us-east-1, we support repos in us-east-2, us-west-1, us-west-2 and eu-west-1. If you use Terraform, we have a module for that. If you’d like us to support another region, drop us a line.
• dynamic-function-loader - This project allows you to dynamically load a Python function (represented as a string) into a running Python script, and use that function as if it was a part of your script. Want to store Python functions in your database and conditionally load them to process something? This makes that easy.
• pg-cognition - Want to use Aurora Serverless, AppSync and Cognito together to create a secure backend for your app and want to ensure row-level security within the database? Then pg-cognition is your tool. We built this for PCI/PHI compliant multi-tenant applications, but it’ll work fine for anyone that wants to robustly secure their data and still be able to use it. Oh - and it’s fast - really fast.

Packer projects

If you’re old-school and still stand up EC2 instances, we have published HashiCorp Packer project to provide hardened Amazon Linux 2 images. The SELinux variants do the baseline hardening, the Managed variants add Inspector, CloudWatch Logs, and SSM.

• amzn2-linux-selinux-packer
• amzn2-linux-managed-packer
• amzn2-eks-selinux-node-packer
• amzn2-eks-managed-node-packer

What’s next

In subsequent posts on this series we will be exploring some of the projects listed above (and others) in more detail, including examples of use. Until then, if you wish to explore, the links above will take you to our various repositories, and the documentation therein is generally pretty good. Enjoy!

Feedback? contact Joe with any comments

About the Author

An experienced C-Level executive with a breadth of experience in operations, sales, marketing, technology, and product development. Joe specializes in innovation and change leadership, having had success as both an entrepreneur and intrapreneur, across numerous industries including defense, insurance, telecommunications, information services and healthcare industries.

Joe relishes the opportunity to make a real difference, both in the success of the corporation and in the lives of the people he interacts with. A self-starter that neither requires nor desires large amounts of oversight having succesfully built numerous DevOps teams and Solution Architected some of the most advanced and secure utility computing applications.