Transformations is one of RudderStack's key features. It gives you the ability to code custom JavaScript functions to implement specific use-cases on your event data, like:

• Filtering/sampling events.
• Enriching events by implementing static logic or leveraging an external API.
• Cleaning/aggregating data.
• Data masking or removing sensitive PII information in the events to ensure data privacy.
• Implementing external actions on the events using an API.

Adding a transformation

• Log into your RudderStack dashboard.
• Click on the Transformations link from the left panel in the dashboard, as shown:

• Click on the Create New option as shown:

• Next, assign a name for this new transformation. Enter the transformation function's code in the Transformation window, as shown:

• Add your transformation code within the transformEvent function in the Transformation window. You can also add other functions and call them from within transformEvent.

• RudderStack also gives you the ability to test your transformation function with the Run Test option as shown:

• In case you want to perform any aggregation/roll-up operation on a micro batch of events, use the transformBatch function, as shown:

Copy
export function transformBatch(events, metadata) {
    return events;
}

Libraries

One of the key features of RudderStack Transformations is the ability to reuse code written for a transformation in other transformations as well. For this, RudderStack lets you create libraries or functions that can be reused in different transformations.

To create a library, follow these steps:

• Click on the Transformations link in the left nav bar and go to the Libraries section. Click on the Create New option to add new libraries, as shown:

• Add the library's Name, an optional Description, and include the custom functions that you need to reuse across all other transformations.

• You can add multiple functions to a single library, as shown:

Using libraries in transformations

To use the libraries in your existing transformation, simply take the name of your library and convert it into camel case without spaces - this becomes your library handle.

For example, if your library name is is rudder email, then the library handle would be isRudderEmail.

Let's say you want to import a function called rudderEmail, which returns true for the emails from a specific domain (e.g. RudderStack) and false otherwise, from the is rudder email library. Also, you want to use this function to filter the events that don't have the email address of the specified domain.

The following code snippet demonstrates how we can implement this use-case:

Copy
import { rudderEmail } from "isRudderEmail";
export function transformEvent(event) {
  const email =
    event.context && event.context.traits && event.context.traits.email;
  if (email) {
    if (!rudderEmail(email)) return;
  }
  return event;
}

On running a test, an example event not having the specified email domain is filtered out, as shown:

Importing multiple functions from a single library

When importing a single function or multiple functions from a library, it is important to keep the import statement to one line. Breaking the import function into multiple lines will lead to an error.

The following snippets highlight how to properly import functions from a library:

Copy
// Correct Way
// ---------------
import { getLoss } from "getFinanceData";

// OR

import { getLoss, getRevenue, getProfit } from "getFinanceData";
import {
  getLoss,
  getRevenue,
  getProfit
} from "getFinanceData";

// For default getPrice import
import getPrice, { getRevenue, getProfit } from "getFinanceData";

// alias imports
import getPrice as gp, { getRevenue as gr, getProfit } from "getFinanceData";
// usuage: gp(event), gr(event), getProfit(ev)

import * as GFD from "getFinanceData";
// usuage: GFD.getRevenue(ev), GFD.getProfit(ev)
// for default import: GFD.default(ev)

Copy
// Incorrect Way
// -----------------
import * from "getFinanceData";

// OR

import getPrice as gp from "getFinanceData";
getPrice(ev)

Accessing metadata

RudderStack injects a function metadata(event) into your transformations as an argument. This allows you to access the event metadata variables that help you customize your transformations.

The following properties, if available, are present in the metadata response:

Since you may not need the metadata in every transformation, you can optionally access it by including it in your function signature as an argument wherever required.

An example of this is as shown below:

Copy
export function transformEvent(event, metadata) {
  const meta = metadata(event);
  event.sourceId = meta.sourceId;

  return event;
}

External API requests

You can make any number of external API requests in your transformation functions and use the response to enrich your events data. RudderStack injects an asynchronous fetch(url) function into your transformations. It makes an API call to the given URL and returns the response in a JSON format.

An example of how to use the fetch function in transformations is shown below:

Copy
export async function transformEvent(event, metadata) {
  const res = await fetch("post_url", {
    method: "POST", // POST, PUT, DELETE, GET, etc.
    headers: {
      "Content-Type": "application/json;charset=UTF-8",
      Authorization: "Bearer <your_authorization_token>"
    },
    body: JSON.stringify(event)
  });
  event.response = JSON.stringify(res);
  return event;
}

FetchV2

FetchV2 is a wrapper around the fetch call to provide more access to fetch the response properties.

The following properties are present in a fetchV2 response:

An example of how to use the fetchV2 function in RudderStack transformations is shown below:

Copy
export async function transformEvent(event) {
  try {
    const res = await fetchV2("url", { timeout: 1000});
    if (res.status == 200) {
      event.response = JSON.stringify(res.body);
    }
  } catch (err) {
    log(err.message);
  }
  return event;
}

Debugging with logs

You can access event-related logs while running a test by including the log() function in your transformation code. An example of this is as shown:

Copy
export function transformEvent(event, metadata) {
  const meta = metadata(event);
  event.sourceId = meta.sourceId;

  log("Event Name is", event.event, ";", "Message Id is", event.messageId);
  log("Source Id is", meta.sourceId);

  return event;
}

On running a test on the above code, you can see the logs in the Logs section of the dashboard, as shown:

Limits

You should take into account the memory and time limits when adding a new transformation. Each invocation of the transformation should not exceed the following limits:

FAQ

Why do I need to use RudderStack's Transformations feature?

RudderStack's Transformations feature lets you transform the events collected from a source into a destination-specific format. Some key benefits of this feature are:

• These transformations can be applied to your in-transit events, in real-time.
• You can implement specific use-cases depending on your business requirements using this feature. Some examples include PII masking, event filtering based on specific criteria, aggregating and enriching events, and so on.
• They're easier to build and manage, and can be reused as well.

I used to write transformations like function transform(events) {}. Why am I not able to create new transformations this way?

We have changed the way transformations are written. RudderStack now supports writing functions that transform a single event instead of a batch of events. You can now define your transformation in the following manner:

export function transformEvent(event, metadata) {}

I want to write a transformation that can be applied to a small batch of events. Can I not do this anymore?

You can. Simply define your transformation in the following manner:

export function transformBatch(events, metadata) {}

I cannot import libraries into my existing transformations. What do I do?

Previously created transformations shown as Version: V0 don't support libraries. If you want to use libraries, please create a new transformation with Version: V1 that support libraries. You can then reconnect your destinations with this new transformation that you have created.

I want to update my existing version v0 transformation code to version v1. What do I do?

You cannot update transformation Version: V0 directly to Version: V1. Please create a new transformation with updated code Version: V1. You can then reconnect your destinations with this new transformation that you have created and delete existing Version: V0 transformation.

Contact us

For queries on any of the sections covered in this guide, you can contact us or start a conversation in our Slack community.