Inspiring Ingenuity

Alteryx, Bicycles and Teaching Kids Programming.

Alteryx: New in 10.1 – HTML SDK

by 6 Comments

With the release of Alteryx 10.1, we’ve added a new software development kit that will enable our users to easily write their own Alteryx tools using web technologies – HTML5, JavaScript and CSS. We think these changes will make Alteryx tool development quicker, simpler and more accessible than ever before – we hope you’ll give our new SDK a try! This is the first in a series of tutorials that will demonstrate how the SDK works.

I’d like to welcome Mike Niland as a co-author.  He is a developer here at Alteryx  and has been working on the HTML SDK feature and graciously offered to help me write a series of posts exploring the new HTML/Javascript SDK.  Read on for more details:

Starting with version 10.1, your copy of Alteryx ships with a hidden reference tool in the bin/HtmlPlugins directory, called JavascriptPluginExample. It provides a template for implementing your own tool. Starting with this template, we’ll demonstrate how to create a simple tool, inspired by the Alteryx Formula tool, which lets you execute JavaScript code on each row of your data. The finished product is available on Github at https://github.com/AlteryxNed/Alteryx_JS_Eval or direct link here.

Note that it’s easiest to build this tool using a non-admin install of Alteryx. In an admin install, the HtmlPlugins directory defaults to read-only, but you’ll want to edit the files in the tool directory during the development process.

Each HTML tool lives in its own folder inside your Alteryx install, under bin/HtmlPlugins. In this case, we’re building a tool named JS_Eval, so it will live in a folder of the same name. The basic folder structure looks like this:

bin/
 HtmlPlugins/
  JS_Eval/
   JS_EvalConfig.xml
   JS_EvalGui.html
   JS_EvalEngine.html
   JS_EvalIcon.png

JS_EvalConfig.xml is the configuration file that tells Alteryx a tool lives in this directory. Its name has to be [Tool Name]Config.xml, and the folder has to be called [Tool Name]; these two names together tell Alteryx what to call the tool in the palette.

The XML config file should look like this:

<?xml version="1.0"?>
<AlteryxJavaScriptPlugin>
  <EngineSettings EngineDll="HTML" EngineDllEntryPoint="JS_EvalEngine.html" SDKVersion="10.1" />
  <GuiSettings Html="JS_EvalGui.html" Icon="JS_EvalIcon.png" SDKVersion="10.1">
    <InputConnections>
      <Connection Name="Input1" AllowMultiple="False" Optional="False" Type="Connection" Label="A"/>
    </InputConnections>
    <OutputConnections>
      <Connection Name="Output" AllowMultiple="False" Optional="True" Type="Connection" Label="B"/>
    </OutputConnections>
  </GuiSettings>
</AlteryxJavaScriptPlugin>

The EngineSettings tag references the JS_EvalEngine.html file in this directory; that file can have any name or live in any subdirectory, as long as the name of the file is also set in this configuration file. The engine HTML file will do the work on your data when you run a workflow that passes data through your tool.

The GuiSettings tag references the JS_EvalGui.html and JS_EvalIcon.png files, which can also have any names, as long as they’re set in this configuration file. The HTML file will be used as the configuration panel for your tool when you click on it in a workflow. The icon is simply the tool’s icon on the canvas and palette.

For this tool, our GUI HTML file is quite simple. We’ll include two text boxes from the Alteryx library of configuration controls. The controls in this library are called plugin widgets; they’re modeled after standard HTML5 input controls, but have additional backend functionality that allows them to save any data entered into them as part of the Alteryx workflow file.

Our GUI HTML will also implement two Javascript functions that are exposed as part of the software development kit. We’ll use AfterLoad to set a default value in the first text box, if it’s blank; we’ll use Annotation to take the contents of the second text box and make that the tool’s annotation on the canvas.

The GUI HTML file is a standard HTML5 file, with the same structure as a normal web page.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <‍script type="text/javascript">
        // Include version 1 of the base GUI library.
        document.write(' + window.Alteryx.LibDir + '1/lib/alteryx/gui/includes.html">');
    <‍/script>
</head>
<body>
    <label for="NewField">New Field Name</label>
    <alteryx-pluginwidget type="TextBox" id="NewField" dataName="NewField"></alteryx-pluginwidget>

    <label for="JsExpresion">JS Expression</label>
    <alteryx-pluginwidget type="TextBox" id="JsExpression" dataName="JsExpression"></alteryx-pluginwidget>

    <‍script type="text/javascript">
        Alteryx.Gui.AfterLoad = function (manager, AlteryxDataItems)
        {
            // set default value
            if (manager.GetDataItem("NewField").value == "") {
                manager.GetDataItem("NewField").setValue("NewField1");
            }
        };

        Alteryx.Gui.Annotation = function (manager)
        {
            return manager.GetDataItem("JsExpression").value;
        };
    <‍/script>
</body>
</html>

The <head> section is boilerplate that must be present in any HTML tool. The first part of the <body> adds the plugin widgets. Alteryx tool HTML documents provide an alteryx-pluginwidget tag, which adds one of the plugin widget controls into the document and allows you to customize it by setting attributes on the tag. The three attributes displayed here are required on every plugin widget. Type determines which kind of control will be placed in the document – the JavascriptPluginExample tool has a complete listing of available widget types. ID becomes the DOM ID of the underlying HTML tag. When the data entered into the widget is saved to the Alteryx document, DataName becomes the name of the XML tag that contains that data.

Finally, we implement the two SDK functions described earlier. The AfterLoad and Annotation functions are provided as part of the SDK, so will automatically be called by the Alteryx base library included at the top of the document. A full list of SDK-provided functions is available in bin/RuntimeData/HtmlAssets/Shared/1/lib/alteryx/gui/main.jsx. AfterLoad is run after the configuration is loaded into the tool: when you click on the tool in the workflow, Alteryx loads your previously-set configuration values from its XML into this HTML document, so that you can manipulate them in the configuration panel. AfterLoad is called after that load step is completed. In this case, it looks in the manager (which is a container for all of the data items being configured in this tool) for the NewField data item – which matches with the dataName of the first text box widget we added to the page. If the value of this text box is blank, the AfterLoad function gives that field a default value.

Annotation is another provided function which creates the canvas annotation – it simply returns the text that will be displayed underneath the tool on the canvas. In this case, it gets JsExpression – the dataName of the second text box – from the manager and returns it.

That’s it for the GUI HTML; one piece is left before we have a complete tool, and that’s the engine HTML. This file exposes a Javascript variation of the standard Alteryx engine API. The engine API is a fairly advanced topic that we’ll continue to cover in more depth as this series continues; today, we’ll implement the three most basic engine functions. These are PI_Init, II_Init, and II_PushRecords. (PI is short for Plugin Interface; II is short for Incoming Interface. The former relates to the tool as a whole; the latter relates to a particular source of input data coming into the tool.)

var TheNewFieldName;
var TheExpression;

Alteryx.Plugin.PI_Init = function(config)
{
    TheNewFieldName = config.Configuration["NewField"];
    TheExpression = config.Configuration["JsExpression"].replace(/\[(.*?)\]/g, "GetFieldValue(\"$1\")");
};

var InputRecordInfo;
var OutputRecordInfo;
var mapFieldNames = {};

Alteryx.Plugin.II_Init = function(metaInfo)
{
    // Save a reference to the RecordInfo passed into this function in the global namespace, so we can access it later.
    InputRecordInfo = metaInfo.RecordInfo;
        
    // Create a new object in the global namespace to contain the metainfo for this tool's output
    OutputRecordInfo = {};
        
    // Populate the OutputRecordInfo object with all of the input fields
    for (var i in InputRecordInfo) {
        OutputRecordInfo[i] = InputRecordInfo[i];
    }

    // Create metainfo for the new field we'll be adding to the end of the output
    var NewField = {};
    NewField.name = TheNewFieldName;
    NewField.type = "V_WString";
    NewField.size = 999999;
        
    // Add that new field's metainfo to the output
    OutputRecordInfo.Field.push(NewField);

    // Create a map that points field names to their index in the OutputRecordInfo's field array
    for (var x = 0; x<OutputRecordInfo.Field.length; ++x)
    {
        mapFieldNames[OutputRecordInfo.Field[x].name] = x;
    }

    // Notify the engine that II_Init has finished processing and send the updated metadata back to the engine
    Alteryx.Engine.SendMessage.RecordInfo("Output", OutputRecordInfo)
};
    
Alteryx.Plugin.II_PushRecords = function(data)
{
    // Create an array to save all the rows of processed output data
    var myRecords = [];
        
    // Create a function that returns the value in this row for a given field name
    var GetFieldValue = function(name)
    {
        if (mapFieldNames[name]==null)
        {
            Alteryx.Engine.SendMessage.Error("The field name \"" + name + "\" is not valid.");
            return;
        }
        return myRow[mapFieldNames[name]];
    }
        
    // Create an error flag to limit us to one error message per run
    var evalError = false;
        
    // Loop through each record (or row) of data that has been passed into this function
    for (var i = 0; i < data.Records.length; i++)
    {
        // Reset the variable that contains this row's output data. Its current contents will be available to GetFieldValue each time GetFieldValue is called.
        var myRow = [];
            
        // Iterate through the fields in this row and add them in order to the output row.
        for (var j = 0; j < data.Records[i].length; j++)
        {
            myRow.push(data.Records[i][j]);
        }

        try
        {
            // Evaluate the expression that the user entered in their second configuration text box. In PI_Init, we searched this expression for instances of [Field1] and
            // replaced them with GetFieldValue("Field1"). Here in II_PushRecords, we've defined the GetFieldValue function, which will look at the current row and give us
            // the value in the specified field of the current row. After evaluating the expression, we push the results into the final field of this row.

            myRow.push(eval(TheExpression));
        }
        catch (e)
        {
            // If there's an error evaluating the Javascript entered by the user, we notify them of it here, set the value of the new field to null,
            // then disable further error messages.
                
            if (!evalError)
                Alteryx.Engine.SendMessage.Error("There was an error in the expression: " +  e.message);
            myRow.push(null);
            evalError= true;
        }
            
        // Push this row onto the end of the complete records array.
        myRecords.push(myRow);
    }
        
    // Signal the engine that II_PushRecords has finished executing, and send the completed array of output data back to the engine.
    Alteryx.Engine.SendMessage.PushRecords("Output", myRecords);
};

The engine calls PI_Init at the beginning of this tool’s lifetime, before it starts to receive input data; this function is simply provided with the configuration data that the user entered in the GUI HTML page. For this tool, our implementation of PI_Init takes in the configuration, prepares the expression entered by the user to be executed as Javascript later in the tool, and saves those pieces of configuration data into the global namespace, so they can be used by the other functions later in the engine’s execution.

TheNewFieldName – a global variable – is simply set to the NewField value that was passed in as part of the config argument to PI_Init. (Note, again, that this is the same dataName that was set for the first plugin widget in the configuration GUI.)

TheExpression is also a global variable. We get the value of JsExpression (the dataName of the second widget), and run a regular expression on it. The regular expression finds any string in between [brackets] and writes that text as an argument to a function: GetFieldValue(“text”). This is simply a replacement in the configured string to make it valid, executable Javascript – once we get the incoming data in a later function, we’ll plug that data into the provided expression and execute it as Javascript.

The next function that will be called is II_Init. It’s called once for each incoming data stream; in this case, the tool has only one input, so II_Init will be called only once. While PI_Init was called without sending any response back to the engine, II_Init is expected to respond with the metadata for each of the fields it will be outputting – types, names and sizes. For this tool, we’ll be outputting the same fields that were given to us as input, plus one new field with the name that the user configured in the first text box – which we earlier saved as TheNewFieldName.

As this function is a little longer, I’ve documented it with inline comments that describe each separate task performed in II_Init. Once II_Init is done, we’ve told the engine what our tool’s output is going to be; all that’s left is to produce the actual output, which is the responsibility of II_PushRecords. Of the three engine functions we’ve implemented in this tool, II_PushRecords is the largest and most complex – I’ll document it with inline comments, as well.

We’ve now completed the final piece of our tool’s engine – we’re ready to start up Alteryx, pull this tool into a workflow, and run it against a live data stream. This example will pull stock quotes as JSON objects from a public REST API.  The tool and sample can be downloaded here, or go to the GitHub repository if that’s how you roll.

6 thoughts on “Alteryx: New in 10.1 – HTML SDK”

Leave a comment

  1. 3danim8 (aka Ken Black)
    December 3, 2015 at 4:54 pm

    Ned,

    You are absolutely going to force me to become a programmer again! I can’t wait to explore this concept to see how crazy I can get. Innovation must be your middle name. Congrats.

    Ken

  2. Jason Mack (@dataMack)
    December 3, 2015 at 9:22 pm

    Really looking forward to following this series. Such forward thinking to give us a framework to extend what’s already the most useful analytics tool available.

    I hope there are plans to somehow incorporate a repository of the tools people start creating in the public Alteryx community/gallery. It will be very helpful for learning plus always inspiration to see how others are implementing ideas.

    • alteryxned
      December 3, 2015 at 9:32 pm

      >I hope there are plans to somehow incorporate a repository of the tools people start creating in the public Alteryx community/gallery.

      It would be silly to do a feature like this and not make the tools easy to distribute, so yes. But one step at a time. It was a lot of work to get this far.

  3. Steve Pasetti
    December 6, 2015 at 10:02 am

    This looks great Ned