In the past 2.5 months, I got to learn a lot of new things. I got to create complex react components; I learned about creating effective UI/UX, I feel I am better at making decisions about product development now. I also enjoyed getting familiar with developing notebook apps using nteract.

As I was learning more about nteract and getting more involved with the community. I noticed that people do show interest in contributing to nteract. However, only a small portion of interested people end up moving forward. What I feel is that there are not many blogs and other resources about the core principle.

In this blog, I will try to demystify nteract and cover nteract SDK in detail. The aim here is to make the engineering behind notebooks simple and more approachable.

Audience

This blog is for anyone interested in either using nteract SDK in their project or someone who wants to contribute to it. There are no hard prerequisites to understand the concept. However, if you are already familiar with Javascript, RxJs, React or Redux, It will be easy for you to understand and get started with it. If you are not familiar with any or some of them, don't worry, the important thing is the willingness to learn. The language for this blog is generic and easy to understand.

nteract organization

Let’s start by looking at the zoomed-out view and see what nteract organization does and look at the overview of its ecosystem.

You can find all the applications and libraries on the nteract's Github page. All the libraries and some applications have their respective repositories with proper documentation for you to get started.

Now let's talk about nteract SDK in-depth, which is used to create interactive notebook applications.

nteract SDK

nteract SDK is a group of javascript packages used to create notebook apps. They all are developed and managed under a mono-repo here. This mono-repo is also used to manage applications based on nteract SDK, like nteract Desktop, nteract Web etc.

There is no official segmentation, but to make it easy to understand, we can divide nteract SDK into three sections.

  • Core Packages
  • UI Packages
  • Jupyter Packages

Core Packages

Core packages provide the basic/core functionality of the notebook. To use core packages, you have to import one package, @nteract/core. This package encapsulates all five core packages. The reason for encapsulating the packages in one is to keep the version the same and reduce errors due to package version mismatch.

You can read about the principle behind core packages here. Five core packages are:

  1. @nteract/types
  2. @nteract/actions
  3. @nteract/reducers
  4. @nteract/selectors
  5. @nteract/epics

@nteract/types

This package contains definitions of different data types used in nteract application. These data types are used to store data for kernels, notebooks, hosts etc.

For example, below is how KernelspecMetadata and KernelspecInfo data types are defined in this package.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

export interface KernelspecMetadata {
  display_name: string;
  language: string;
  argv: string[];
  name?: string;
  env?: {
    [variable: string]: string;
  };
}

export interface KernelspecInfo {
  name: string;
  spec: KernelspecMetadata;
}

@nteract/actions

This package contains definitions of constants and action creators that can be used to create Redux actions in nteract application. These actions can be dispatched to create new cells, launch kernels, create new notebooks, execute cells, and much more.

For example, below is how you define an action to update kernel specifications. Also note it uses the `KernelspecInfo` data type as parameters(payload), which is defined above. 

1
2
3
4
5
6
7
8

// Define the action type
export const SET_KERNELSPEC_INFO   = "SET_KERNELSPEC_INFO";

// Define the payload
export type SetKernelspecInfo = Action <typeof SET_KERNELSPEC_INFO,  HasContent & { kernelInfo: KernelspecInfo }>;

// Create action
export const setKernelspecInfo  = makeActionFunction <SetKernelspecInfo> (SET_KERNELSPEC_INFO);

@nteract/reducers

This package contains a set of Redux reducers for use in nteract applications. Reducers describe the change in the application's state in response to actions sent to the store.

Below is how reducer sets the kernel information when `SetKernelspecInfo` action is dispatched. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

...
case actionTypes.SET_KERNEL_INFO:
      typedAction = action as actionTypes.SetKernelInfo;
      let codemirrorMode = typedAction.payload.info.codemirrorMode;
      if (!codemirrorMode) {
          codemirrorMode = typedAction.payload.info.languageName;
      }

         switch (typeof codemirrorMode) {
                 case "string":
                    // already set as we want it
                 break;
                 case "object":
                    codemirrorMode = Map(codemirrorMode as JSONObject);
                 break;
                 default:
                    // any other case results in falling back to language name
                    codemirrorMode = typedAction.payload.info.languageName;
           }

        const helpLinks = typedAction.payload.info.helpLinks
                ? List(
                    (typedAction.payload.info.helpLinks as HelpLink[]).map(
                        makeHelpLinkRecord
                    )
                  )
            : List();

        return state.setIn(
                [typedAction.payload.kernelRef, "info"],
                makeKernelInfoRecord(typedAction.payload.info).merge({
                  helpLinks,
                  codemirrorMode
                })
              );
...

@nteract/selectors

This package provides a set of selectors and functions that allow you to extract important information from the state of your nteract application. To see a full set of the data stored in application state that be extracted with this package, you can view the AppState type.

Below is how the currentKernel selector is defined.

1
2
3
4
5
6
7
8

/**
 * Returns the kernelspec of the kernel that we are currently connected to.
 * Returns null if there is no kernel.
 */
export const currentKernel = createSelector(
  [currentKernelRef, kernelsByRef],
  (kernelRef, byRef) => (kernelRef ? byRef.get(kernelRef) : null)
);

Below is how we use the selector in epics or in UI library

1
2
3
4

import * as selectors from "@nteract/selectors";
...
kernel = selectors.currentKernel(state$.value);
...

@nteract/epics

This package contains a set of Redux-Observable epics for use in nteract applications. It is a function which takes a stream of actions and returns a stream of actions.

Below is how acquireKernelInfoEpic is defined.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

/**
 * Gets information about newly launched kernel.
 *
 * @param  {ActionObservable}  The action type
 */
 export const acquireKernelInfoEpic = (
    action$: Observable<actions.NewKernelAction>,
    state$: StateObservable<AppState>
 ) =>
  action$.pipe(
    ofType(actions.LAUNCH_KERNEL_SUCCESSFUL),
    switchMap((action: actions.NewKernelAction) => {
      const {
         payload: {
            kernel: { channels, kernelSpecName },
            kernelRef,
            contentRef
          }
       } = action;
      
      return acquireKernelInfo(
         channels,
         kernelRef,
         contentRef,
         state$.value,
         kernelSpecName
      );
    })
 );

UI Packages

UI Packages are react based components used to created different UI elements of the nteract ecosystems like notebook, cells, buttons etc. Some of the packages are stateful, but some are not and require you to set up your store. If you want to help with improving the UI of nteract, these are the packages you can explore.

Jupyter Packages

Jupyter packages help in interacting with different components of Jupyter ecosystem, like jupyter server, binder etc. Following are the jupyter packages in nteract ecosystem.

Conclusion

I hope this blog has given you some more confidence in getting started with nteract SDK and contribute to it. If you are planning to use it for development, this blog should be a good starting point in understanding the working of it. I will suggest you read more about it on Github and join the community on Slack </p.

Acknowledgment