Contents

The Dynamic Function Loader - Open Source Part 2

There are often times that, as a software engineer, I want to load a function to allow for dynamic behavior in a piece of software that I’m developing. For instance, I may want to allow the users of my software to store discrete function code in a database and then dynamically load that code and use it based upon the selected database record.

For performance reasons, once that function is loaded in my program I want it to behave in my software as if it were an integral part of my software and not something loaded from an external source.

Given that I do most of my work in Python and Python is an interpreted language, you would think that this would be relatively easy to do. Well, not so much.

Executing external Python code is relatively simple. But dynamically creating an actual function that can be called repetitively and be treated by Python as a normal function is far more difficult (until you know how…)!

I’m a firm believer in creating a reuable library for things that I do more than twice. Also, I’m not a young as I used to be, and a short pencil (or a short library) beats a long memory every day.

Enter the dynamic-function-loader python library.

Installing the dynamic-function-loader

dynamic-function-loader is a pip package, so installation is as simple as

pip install dynamic-function-loader

Use

First we’ll need to create a function to load. This can be any Python function definition, with the following caveat; any libraries that are required by the function must be imported within the function definition.

Note - any libraries available in the loading Python script will be available in the loaded function!

test_function.py:

def test_function(foo):
    import json

    bar = json.loads(foo)

    print(bar)

Next, we’ll need to load that function in our parent script. Once the function is loaded, we can use it as if it was written within the script itself.

test_script.py:

import dynamic_function_loader

test_function_string = ""

with open("test_function.py") as tf:
    test_function_string = tf.read()

my_func = dynamic_function_loader.load(test_function_string)

foo = '{"a": 1}'

my_func(foo)

This will run test_function (named my_func) in our script and produce:

> {'a': 1}

Access to script globals

Any function loaded in this manner will be part of the interpreter that loads it. As such, it will not be sandboxed to its own interpreter and will have access to the loading script’s globals and system permissions, just like any other function in the script.

This is both an great thing and a potentially dangerous thing. It becomes dangerous if you do not have control over the environment the primary script is running in and you do not have direct control of the function code that is being loaded.

For example, running the loading script as root while accepting functions to load from the web is the definition of a code injection vulnerability and should never be done.

Caveat emptor…


Feedback? contact Joe with any comments

About the Author

An experienced C-Level executive with a breadth of experience in operations, sales, marketing, technology, and product development. Joe specializes in innovation and change leadership, having had success as both an entrepreneur and intrapreneur, across numerous industries including defense, insurance, telecommunications, information services and healthcare industries.

Joe relishes the opportunity to make a real difference, both in the success of the corporation and in the lives of the people he interacts with. A self-starter that neither requires nor desires large amounts of oversight having succesfully built numerous DevOps teams and Solution Architected some of the most advanced and secure utility computing applications.