When it comes to error management, there are three things that I've been thinking about recently that I really like:

In particular with EffectTS, I think their approach to error handling is a fantastic foundation for writing scalable TypeScript.

At my company, I've run into some recent problems that I think EffectTS would solve quite nicely. However, achieving buy-in for it at this point in time is difficult (I won't elaborate on the reasons) and in earnest, I am only hoping to reap the benefits of a few adopted features at this point in time.

So, I've been thinking about how I could implement my own Result type in TypeScript that could nicely return "successes" or "failures" that gives me nice TypeScript intellisense and type checking around errors. The aim would be to have something relatively simple that could be easily adopted at work, but also easily removed if need be.

In this blog post, we will take a look at libraries such as EffectTS, Zod, Valibot and Joi to understand how they approach a Result-like type and how we could approach building our own.

As far as I am talking about, I am looking to write a Result type is a type that can represent either a success or a failure.

I want the success type to return data (hopefully via a property like data) and a way to known it's a success, while the failure can safely return an error without throwing an exception.

The aim for failure types is that they will represent the "expected errors" i.e. errors that we expect could happen. This gives me power over how I handle these errors and how I can recover from them. It also can give me great type checking around these errors to improve developer experience.

Anything unexpected (also known as a defect in EffectTS) would be the errors that do end up in the catch part of the try/catch block. My aim here is that this is the stuff that we pay close attention to and aim to patch when these unexpected errors show their face.

Originally, I had a very simple implementation of a Result type that looked like this:

export type Failure<E = unknown> = {
  _tag: "Failure";
  error: E;
};

export type Success<T = void> = {
  _tag: "Success";
  data: T;
};

export type Result<T = void, E = unknown> = Failure<E> | Success<T>;

export function succeed<T = void>(data: T): Success<T> {
  return {
    _tag: "Success",
    data: data,
  };
}

export function fail<E = unknown>(error: E): Failure<E> {
  return {
    _tag: "Failure",
    error: error,
  };
}
copy

I figured that the Result type would be a union of Failure and Success types, which I could create with the fail and succeed functions:

const success = succeed("Hello, world!"); // Success<string>
const failure = fail("Something went wrong"); // Failure<string>
copy

That being said, it felt super basic and like it was missing some stuff that I wanted, so I decided to snoop into some source code and see how the pros do it.

I started with Zod due to the amount of usage I spend with it on my personal work.

I use safeParse and safeParseAsync frequently, and the API looks little like the following:

const result = stringSchema.safeParse("billie");
if (!result.success) {
  // handle error then return
  result.error;
} else {
  // do something
  result.data;
}
copy

The API is very simple and intuitive, so it gets brownie points from me.

Looking into the source code for safeParse and safeParseAsync, both we returning Promise<SafeParseReturnType<Input, Output>> which is the return type of the handleResult function.

Going further into the handleResult source code, we look to work with the following:

const handleResult = <Input, Output>(
  ctx: ParseContext,
  result: SyncParseReturnType<Output>
):
  | { success: true; data: Output }
  | { success: false; error: ZodError<Input> } => {
  if (isValid(result)) {
    return { success: true, data: result.value };
  } else {
    if (!ctx.common.issues.length) {
      throw new Error("Validation failed but no issues detected.");
    }

    return {
      success: false,
      get error() {
        if ((this as any)._error) return (this as any)._error as Error;
        const error = new ZodError(ctx.common.issues);
        (this as any)._error = error;
        return (this as any)._error;
      },
    };
  }
};
copy

The unsuccessful route returns a ZodError, which both the docs and ZodError source code show that it is a class that extends Error.

Although it makes perfect sense for Zod's use case, I'm hoping to return a more generic class for my expected errors that do not extend Error. There reason being is that I don't want those errors to be throwable.

Next, I took a look at Joi, another validation library which is the one heavily used at my current company.

The validate function from Joi is what runs the assertions:

const Joi = require("joi");

const schema = Joi.object({
  username: Joi.string().alphanum().min(3).max(30).required(),

  password: Joi.string().pattern(new RegExp("^[a-zA-Z0-9]{3,30}$")),

  repeat_password: Joi.ref("password"),

  access_token: [Joi.string(), Joi.number()],

  birth_year: Joi.number().integer().min(1900).max(2013),

  email: Joi.string().email({
    minDomainSegments: 2,
    tlds: { allow: ["com", "net"] },
  }),
})
  .with("username", "birth_year")
  .xor("password", "access_token")
  .with("password", "repeat_password");

schema.validate({ username: "abc", birth_year: 1994 });
// -> { value: { username: 'abc', birth_year: 1994 } }

schema.validate({});
// -> { value: {}, error: '"username" is required' }

// Also -

try {
  const value = await schema.validateAsync({
    username: "abc",
    birth_year: 1994,
  });
} catch (err) {}
copy

Checking the validate source code, it uses the hapijs/hoek libraries assert function under the hood.

That too makes use of a AssertError class that extends Error:

"use strict";

const internals = {};

module.exports = class AssertError extends Error {
  name = "AssertError";

  constructor(message, ctor) {
    super(message || "Unknown error");

    if (typeof Error.captureStackTrace === "function") {
      // $lab:coverage:ignore$
      Error.captureStackTrace(this, ctor);
    }
  }
};
copy

Valibot was the last assertion library that I checked out. I personally haven't used Valibot much, but I've heard the hype.

Similar to Zod, Valibot has a safeParse function:

const EmailSchema = v.pipe(v.string(), v.email());
const result = v.safeParse(EmailSchema, "jane@example.com"); // SafeParseResult<TSchema>

if (result.success) {
  const email = result.output; // string
} else {
  console.log(result.issues);
}
copy

Issues look to be related to the BaseIssue interface, as well as making heavy use of the InferIssue utility function.

The type for SafeParseResult looks like the following:

import type {
  BaseIssue,
  BaseSchema,
  BaseSchemaAsync,
  InferIssue,
  InferOutput,
} from "../../types/index.ts";

/**
 * Safe parse result type.
 */
export type SafeParseResult<
  TSchema extends
    | BaseSchema<unknown, unknown, BaseIssue<unknown>>
    | BaseSchemaAsync<unknown, unknown, BaseIssue<unknown>>
> =
  | {
      /**
       * Whether is's typed.
       */
      readonly typed: true;
      /**
       * Whether it's successful.
       */
      readonly success: true;
      /**
       * The output value.
       */
      readonly output: InferOutput<TSchema>;
      /**
       * The issues if any.
       */
      readonly issues: undefined;
    }
  | {
      readonly typed: true;
      readonly success: false;
      readonly output: InferOutput<TSchema>;
      readonly issues: [InferIssue<TSchema>, ...InferIssue<TSchema>[]];
    }
  | {
      readonly typed: false;
      readonly success: false;
      readonly output: unknown;
      readonly issues: [InferIssue<TSchema>, ...InferIssue<TSchema>[]];
    };
copy

Interestingly, the BaseIssue interface does not look to extend from Error:

/**
 * Base issue type.
 */
export interface BaseIssue<TInput> extends Config<BaseIssue<TInput>> {
  /**
   * The issue kind.
   */
  readonly kind: "schema" | "validation" | "transformation";
  /**
   * The issue type.
   */
  readonly type: string;
  /**
   * The raw input data.
   */
  readonly input: TInput;
  /**
   * The expected property.
   */
  readonly expected: string | null;
  /**
   * The received property.
   */
  readonly received: string;
  /**
   * The error message.
   */
  readonly message: string;
  /**
   * The input requirement.
   */
  readonly requirement?: unknown;
  /**
   * The issue path.
   *
   * TODO: Investigate if it is possible to make the path type safe based on the
   * input.
   */
  readonly path?: [IssuePathItem, ...IssuePathItem[]];
  /**
   * The sub issues.
   */
  readonly issues?: [BaseIssue<TInput>, ...BaseIssue<TInput>[]];
}
copy

Finally, I went to check out what EffectTS does given how much their paradigm had inspired my initial search.

EffectTS has a Success and Error representation within the Effect namespace.

We can create a "Success" representation by invoking the Effect.succeed function:

import { Effect } from "@effect-ts/core";

const success = Effect.succeed("Hello, world!"); // Effect<string, never>
copy

Although this represents a success, the type is in fact still Effect.

The source code for succeed is interesting (and a little trickier to grok):

/* @internal */
export const succeed = <A>(value: A): Effect.Effect<A> => {
  const effect = new EffectPrimitiveSuccess(OpCodes.OP_SUCCESS) as any;
  effect.effect_instruction_i0 = value;
  return effect;
};
copy

Looking into EffectPrimitiveSuccess led me to the alternative types EffectPrimitive and EffectPrimitiveFailure, where each look to have a similar implementation:

/** @internal */
class EffectPrimitiveSuccess {
  public effect_instruction_i0 = undefined;
  public effect_instruction_i1 = undefined;
  public effect_instruction_i2 = undefined;
  public trace = undefined;
  [EffectTypeId] = effectVariance;
  constructor(readonly _op: Primitive["_op"]) {
    // @ts-expect-error
    this._tag = _op;
  }
  [Equal.symbol](this: {}, that: unknown) {
    return (
      exitIsExit(that) &&
      that._op === "Success" &&
      // @ts-expect-error
      Equal.equals(this.effect_instruction_i0, that.effect_instruction_i0)
    );
  }
  [Hash.symbol](this: {}) {
    return pipe(
      // @ts-expect-error
      Hash.string(this._tag),
      // @ts-expect-error
      Hash.combine(Hash.hash(this.effect_instruction_i0)),
      Hash.cached(this)
    );
  }
  get value() {
    return this.effect_instruction_i0;
  }
  pipe() {
    return pipeArguments(this, arguments);
  }
  toJSON() {
    return {
      _id: "Exit",
      _tag: this._op,
      value: toJSON(this.value),
    };
  }
  toString() {
    return format(this.toJSON());
  }
  [NodeInspectSymbol]() {
    return this.toJSON();
  }
  [Symbol.iterator]() {
    return new SingleShotGen(new YieldWrap(this));
  }
}
copy

I won't explain too much of this (since it is a tough for me to explain with 100% confidence), but my big assumption here is that the implementation here is required for some important work under the hood, and so for my use case I am likely looking to use something that is simpler in terms of naming conventions, but some of the symbol properties do look important for me to implement.

Although it's a bit of off scope, something else that is really impressive about EffectTS on top of this though is how they make use of readonly tags. This helps with their conditional logic a lot, and I am a fan of some of their patterns that make use of _tag. I am hoping to make use of this myself in the final result.

After reviewing all of the different approaches, I ended up with the following implementation for Result:

export type Success<T> = Result<T, never>;
export type Failure<E extends ErrorType> = Result<never, E>;

export type ErrorType = {
  _tag: string;
  [key: string]: any;
};

type MatchCases<T, E extends ErrorType, U> = {
  Success: (data: T) => U;
} & {
  [K in E["_tag"]]: (error: Extract<E, { _tag: K }>) => U;
};

export class Result<T, E extends ErrorType> {
  protected constructor(
    readonly _tag: "Success" | "Failure",
    protected readonly value: T | E
  ) {}

  static succeed<T>(data: T): Success<T> {
    return new Result("Success", data) as Success<T>;
  }

  static fail<E extends ErrorType>(error: E): Failure<E> {
    return new Result("Failure", error) as Failure<E>;
  }

  isSuccess(): this is Success<T> {
    return this._tag === "Success";
  }

  isFailure(): this is Failure<E> {
    return this._tag === "Failure";
  }

  get data(): T {
    if (this.isSuccess()) return this.value as T;
    throw new Error("Cannot get data from a Failure");
  }

  get error(): E {
    if (this.isFailure()) return this.value as E;
    throw new Error("Cannot get error from a Success");
  }

  map<U>(f: (value: T) => U): Result<U, E> {
    return this.isSuccess()
      ? Result.succeed(f(this.data))
      : (this as unknown as Result<U, E>);
  }

  flatMap<U>(f: (value: T) => Result<U, E>): Result<U, E> {
    return this.isSuccess() ? f(this.data) : (this as unknown as Result<U, E>);
  }

  equals(that: unknown): boolean {
    return (
      that instanceof Result &&
      this._tag === that._tag &&
      this.value === that.value
    );
  }

  toJSON() {
    return {
      _tag: this._tag,
      [this._tag === "Success" ? "data" : "error"]: this.value,
    };
  }

  toString(): string {
    return JSON.stringify(this.toJSON());
  }

  [Symbol.for("nodejs.util.inspect.custom")]() {
    return this.toJSON();
  }

  static matchTag<T, E extends ErrorType, U>(
    result: Result<T, E>,
    cases: MatchCases<T, E, U>
  ): U {
    if (result.isSuccess()) {
      return cases.Success(result.data);
    } else {
      const errorHandler = cases[result.error._tag as keyof typeof cases];
      if (errorHandler) {
        return errorHandler(result.error as any);
      }
      throw new Error(`Unhandled error type: ${result.error._tag}`);
    }
  }
}
copy

Let's break down the code.

Definitions and Types

Result Class

Summary

As you can tell, this does take a lot of inspiration from the EffectTS implementation, but I've tried to simplify it down to the core features that I need as well as using the other validation libraries as inspiration.

This approach allows for clear and type-safe handling of operations that may succeed or fail, providing strong guarantees about the types involved.

type TestError =
  | { _tag: "TestError1"; message: string }
  | { _tag: "TestError2"; message: string };

const result = Result.succeed(42); // Result<T, E extends ErrorType>.succeed<number>(data: number): Success<number>

// Result<T, E extends ErrorType>.fail<{
//     _tag: "TestError1";
//     message: string;
// }>(error: {
//     _tag: "TestError1";
//     message: string;
// }): Failure<{
//     _tag: "TestError1";
//     message: string;
// }>
const failure = Result.fail<TestError>({
  _tag: "TestError1",
  message: "Test error",
});
copy

The best way to get a feel for how this works is to look at the unit tests that I wrote for this type:

import { Result, ErrorType } from "./alt-result";

describe("Result", () => {
  // Define some test error types
  type TestError =
    | { _tag: "TestError1"; message: string }
    | { _tag: "TestError2"; message: string };

  describe("creation and basic methods", () => {
    it("should create a Success result", () => {
      const result = Result.succeed(42);
      expect(result.isSuccess()).toBe(true);
      expect(result.isFailure()).toBe(false);
      expect(result.data).toBe(42);
    });

    it("should create a Failure result", () => {
      const error: TestError = { _tag: "TestError1", message: "Test error" };
      const result = Result.fail(error);
      expect(result.isSuccess()).toBe(false);
      expect(result.isFailure()).toBe(true);
      expect(result.error).toEqual(error);
    });

    it("should throw when accessing data of a Failure", () => {
      const result = Result.fail({ _tag: "TestError1", message: "Test error" });
      expect(() => result.data).toThrow("Cannot get data from a Failure");
    });

    it("should throw when accessing error of a Success", () => {
      const result = Result.succeed(42);
      expect(() => result.error).toThrow("Cannot get error from a Success");
    });
  });

  describe("map and flatMap", () => {
    it("should map a Success result", () => {
      const result = Result.succeed(42).map((x) => x * 2);
      expect(result.isSuccess()).toBe(true);
      expect(result.data).toBe(84);
    });

    it("should not map a Failure result", () => {
      const error: TestError = { _tag: "TestError1", message: "Test error" };
      const result = Result.fail<TestError>(error).map((x) => x * 2);
      expect(result.isFailure()).toBe(true);
      expect(result.error).toEqual(error);
    });

    it("should flatMap a Success result", () => {
      const result = Result.succeed(42).flatMap((x) => Result.succeed(x * 2));
      expect(result.isSuccess()).toBe(true);
      expect(result.data).toBe(84);
    });

    it("should not flatMap a Failure result", () => {
      const error: TestError = { _tag: "TestError1", message: "Test error" };
      const result = Result.fail<TestError>(error).flatMap((x) =>
        Result.succeed(x * 2)
      );
      expect(result.isFailure()).toBe(true);
      expect(result.error).toEqual(error);
    });
  });

  describe("equals", () => {
    it("should consider two Success results with the same data equal", () => {
      const result1 = Result.succeed(42);
      const result2 = Result.succeed(42);
      expect(result1.equals(result2)).toBe(true);
    });

    it("should consider two Failure results with the same error equal", () => {
      const error: TestError = { _tag: "TestError1", message: "Test error" };
      const result1 = Result.fail(error);
      const result2 = Result.fail(error);
      expect(result1.equals(result2)).toBe(true);
    });

    it("should consider Success and Failure results not equal", () => {
      const success = Result.succeed(42);
      const failure = Result.fail({
        _tag: "TestError1",
        message: "Test error",
      });
      expect(success.equals(failure)).toBe(false);
    });
  });

  describe("matchTag", () => {
    it("should match Success case", () => {
      const result = Result.succeed(42);
      const output = Result.matchTag(result as Result<number, TestError>, {
        Success: (data) => `Success: ${data}`,
        TestError1: (error: TestError) => `Error1: ${error.message}`,
        TestError2: (error: TestError) => `Error2: ${error.message}`,
      });
      expect(output).toBe("Success: 42");
    });

    it("should match Failure case", () => {
      const result = Result.fail<TestError>({
        _tag: "TestError1",
        message: "Test error",
      });
      const output = Result.matchTag(result, {
        Success: (data) => `Success: ${data}`,
        TestError1: (error: TestError) => `Error1: ${error.message}`,
        TestError2: (error: TestError) => `Error2: ${error.message}`,
      });
      expect(output).toBe("Error1: Test error");
    });

    it("should throw for unhandled error types", () => {
      const result = Result.fail({ _tag: "UnhandledError" } as ErrorType);
      expect(() =>
        Result.matchTag(result as unknown as Result<number, TestError>, {
          Success: (data) => `Success: ${data}`,
          TestError1: (error: TestError) => `Error1: ${error.message}`,
          TestError2: (error: TestError) => `Error2: ${error.message}`,
        })
      ).toThrow("Unhandled error type: UnhandledError");
    });
  });
});
copy

After playing around a bit more, I've done some refactoring. Right now, my current implementation looks like this:

export type Success<T> = Result<T, never>;
export type Failure<E> = Result<never, E>;

/**
 * !!! EXPERIMENTAL !!!
 * The following class provides a way to enapsulate the result of an operation that can either succeed or fail.
 * It enables the use of functional programming techniques to handle the result of an operation. This can
 * be useful in TypeScript for handling expected errors in a more type-safe way.
 *
 * @experimental
 * @see https://blog.dennisokeeffe.com/blog/2024-07-14-creating-a-result-type-in-typescript
 *
 * @example
 * type TestError =
 *  | { _tag: "TestError1"; message: string }
 *  | { _tag: "TestError2"; message: string };
 *
 *  // Result<T, E extends ErrorType>.succeed<number>(data: number): Success<number>
 * const result = Result.succeed(42);
 *
 * // Result<T, E extends ErrorType>.fail<{
 * //     _tag: "TestError1";
 * //     message: string;
 * // }>(error: {
 * //     _tag: "TestError1";
 * //     message: string;
 * // }): Failure<{
 * //     _tag: "TestError1";
 * //     message: string;
 * // }>
 * const failure = Result.fail<TestError>({
 * _tag: "TestError1",
 *  message: "Test error",
 * });
 */
export class Result<T, E> {
  protected constructor(
    readonly _tag: "Success" | "Failure",
    protected readonly value: T | E
  ) {}

  static succeed<T>(data: T): Success<T> {
    return new Result("Success", data) as Success<T>;
  }

  static fail<E>(error: E): Failure<E> {
    return new Result("Failure", error) as Failure<E>;
  }

  isSuccess(): this is Success<T> {
    return this._tag === "Success";
  }

  isFailure(): this is Failure<E> {
    return this._tag === "Failure";
  }

  get data(): T {
    if (this.isSuccess()) return this.value as T;
    throw new Error("Cannot get data from a Failure");
  }

  get error(): E {
    if (this.isFailure()) return this.value as E;
    throw new Error("Cannot get error from a Success");
  }

  map<U>(f: (value: T) => U): Result<U, E> {
    return this.isSuccess()
      ? Result.succeed(f(this.data))
      : (this as unknown as Result<U, E>);
  }

  flatMap<U>(f: (value: T) => Result<U, E>): Result<U, E> {
    return this.isSuccess() ? f(this.data) : (this as unknown as Result<U, E>);
  }

  equals(that: unknown): boolean {
    return (
      that instanceof Result &&
      this._tag === that._tag &&
      this.value === that.value
    );
  }

  toJSON() {
    return {
      _tag: this._tag,
      [this._tag === "Success" ? "data" : "error"]: this.value,
    };
  }

  toString(): string {
    return JSON.stringify(this.toJSON());
  }

  [Symbol.for("nodejs.util.inspect.custom")]() {
    return this.toJSON();
  }
}
copy

There is also another approach I am mulling over to reduce some as casting for the following:

static succeed<T>(data: T): Success<T> {
  return new Result("Success", data) as Success<T>;
}

static fail<E>(error: E): Failure<E> {
  return new Result("Failure", error) as Failure<E>;
}
copy

That approach would be to convert Result to an abstract class and convert the Success and Failure types into sub classes:

export abstract class Result<T, E> {
  protected constructor(
    readonly _tag: "Success" | "Failure",
    protected readonly value: T | E
  ) {}

  static succeed<T>(data: T): Success<T> {
    return new Success(data);
  }

  static fail<E>(error: E): Failure<E> {
    return new Failure(error);
  }

  abstract isSuccess(): this is Success<T>;
  abstract isFailure(): this is Failure<E>;

  get data(): T {
    if (this.isSuccess()) return this.value;
    throw new Error("Cannot get data from a Failure");
  }

  get error(): E {
    if (this.isFailure()) return this.value;
    throw new Error("Cannot get error from a Success");
  }

  map<U>(f: (value: T) => U): Result<U, E> {
    return this.isSuccess()
      ? Result.succeed(f(this.data))
      : (this as unknown as Result<U, E>);
  }

  flatMap<U>(f: (value: T) => Result<U, E>): Result<U, E> {
    return this.isSuccess() ? f(this.data) : (this as unknown as Result<U, E>);
  }

  equals(that: unknown): boolean {
    return (
      that instanceof Result &&
      this._tag === that._tag &&
      this.value === that.value
    );
  }

  toJSON() {
    return {
      _tag: this._tag,
      [this._tag === "Success" ? "data" : "error"]: this.value,
    };
  }

  toString(): string {
    return JSON.stringify(this.toJSON());
  }

  [Symbol.for("nodejs.util.inspect.custom")]() {
    return this.toJSON();
  }
}

export class Success<T> extends Result<T, never> {
  constructor(data: T) {
    super("Success", data);
  }

  isSuccess(): this is Success<T> {
    return true;
  }

  isFailure(): this is Failure<never> {
    return false;
  }

  get data(): T {
    return this.value;
  }
}

export class Failure<E> extends Result<never, E> {
  constructor(error: E) {
    super("Failure", error);
  }

  isSuccess(): this is Success<never> {
    return false;
  }

  isFailure(): this is Failure<E> {
    return true;
  }

  get error(): E {
    return this.value;
  }
}
copy

Both approaches should only require removal of the matchTag tests that I had previously.

In a recent data migration, I ended up making use of a Result-ish type again to help capture and propagate any expected errors for my logging to help decide what I needed to do.

I won't go into too many details for the script itself, but in that approach I ended up stripping everything back to a simple set of types and functions:

class Ok<T> {
  readonly _tag = "ok";
  value: T;

  constructor(value: T) {
    this.value = value;
  }
}

class Err<E> {
  readonly _tag = "err";
  error: E;

  constructor(error: E) {
    this.error = error;
  }
}

type Result<T, E> = Ok<T> | Err<E>;

function err<E>(e: E): Err<E> {
  return new Err(e);
}

function ok<T>(t: T): Ok<T> {
  return new Ok(t);
}
copy

This approach omits a lot of the helper functions that I had for the classes, but it felt like a good balance for the use-case that I had. There can be issues with serialization of the errors for the generic passed in, but it relies a lot more on the call site invocation to handle that.

In use, you could set an expected error and then handle it accordingly:

class ExpectedError {
  readonly _tag = "ExpectedError";
}

function resultTest(): Result<string, ExpectedError> {
  const run = true;

  if (run) {
    return err(new ExpectedError());
  }

  return ok("Success");
}

const res2 = resultTest();

switch (res2._tag) {
  case "ok":
    console.log(res2.value);
    break;
  case "err":
    switch (res2.error._tag) {
      case "ExpectedError":
        console.log("Handle something here");
        break;
    }
    break;
}
copy

Triggering the ExpectedError within that contrived code demonstrates the use case. It's a bit contrived here with the switch-within-a-switch, but that's purely for demonstration purposes. In reality, you would make use of a more well-thought-out set of conditional logic.

In my particular use-case, I had a number of Node.js streams where I needed to propagate errors up the chain and log them accordingly. An example of one such stream where I needed to bulk insert into a DynamoDB table at that part of the pipeline:

class UnprocessedItemsError {
  readonly _tag = "UnprocessedItemsError";
  readonly barcode: string;

  constructor(barcode: string) {
    this.barcode = barcode;
  }
}

class DynamoDBError {
  readonly _tag = "DynamoDBError";
  readonly barcode: string;

  constructor(barcode: string) {
    this.barcode = barcode;
  }
}

type BarcodeObj = {
  barcode: string;
};

export async function* processBatchIntoFactTelling(
  source: AsyncIterable<ProcessBatchIntoBaasResult[]>,
  tableName: string,
  client: DynamoDBClient,
  uploadId: string,
  actioner: Actioner,
  walletId: string
): AsyncGenerator<Result<BarcodeObj, UnprocessedItemsError | DynamoDBError>[]> {
  const BATCH_SIZE = 25;

  for await (const batch of source) {
    const unwrapped = batch
      .filter((item) => item._tag === "ok")
      .map((item) => item.value);
    const results: Result<BarcodeObj, UnprocessedItemsError | DynamoDBError>[] =
      [];

    try {
      for (let i = 0; i < unwrapped.length; i += BATCH_SIZE) {
        const writeRequests: any[] = [];
        const currentChunk = unwrapped.slice(i, i + BATCH_SIZE);

        currentChunk.forEach(() => {
          const timestampISOString = getSequentialTimestamp();
          writeRequests.push({
            PutRequest: {
              Item: marshall({
                // ... omitted DynamoDB item
              }),
            },
          });
        });

        const command = new BatchWriteItemCommand({
          RequestItems: {
            [tableName]: writeRequests,
          },
        });

        const response = await client.send(command);

        if (
          response.UnprocessedItems &&
          response.UnprocessedItems[tableName] &&
          response.UnprocessedItems[tableName].length > 0
        ) {
          results.push(
            ...currentChunk.map((barcodeObj) =>
              err(new UnprocessedItemsError(barcodeObj.barcode))
            )
          );
        } else {
          results.push(...currentChunk.map((item) => ok(item)));
        }
      }

      yield results;
    } catch (error) {
      console.log("DynamoDB error", error);
      yield unwrapped.map((item) => err(new DynamoDBError(item.barcode)));
    }
  }
}
copy

This isn't as specific as some of the other streams, but in this particular case I was interested in unprocessed items or general failures.

I also had another function that helped to split the stream by the take so I could write the logs into two separate files to prepare for retries:

export const splitByTag = <T, U>(array: (Ok<T> | Err<U>)[]) => {
  const ok: Ok<T>[] = [];
  const err: Err<U>[] = [];

  return array.reduce(
    (result, item) => {
      const { ok, err } = result;

      if (item._tag === "ok") {
        return { ...result, ok: [...ok, item] };
      } else {
        return { ...result, err: [...err, item] };
      }
    },
    { ok, err }
  );
};
copy

That same _tag property could be applied further for making decisions on what to do for specific errors and help type narrow down even further. It was very nice to work with.

I also ended up implementing a similar Some type inspired by Rust which I might go into detail in for another post which worked well in representing optional values that could also apply the same conditional logic thanks to the _tag property instead of using something like undefined. Once again though, although the developer experience of using this feels enhanced, it's an abstraction that likely comes at a big cost if performance is the number one metric, but I haven't bothered with any benchmarks. It was mostly about having code that was easy to pick up again and reason with.

At the moment, I am content with this implementation. As you'll see in some upcoming posts, it balances the complexity of the implementation with it's use-case power.

Please note that this is still early days and a little raw, which the end-game meaning to be a useful utility that might be a stepping stone towards something more powerful.

If you find some obvious improvements or have some feedback, please feel free to reach out to me on Twitter. I would be very keen to hear your thoughts.

Disclaimer: This blog post used AI to generate the images used for the analogy.

Photo credit: pawel_czerwinski