Introduction to ScalaPy

1. Introduction

In this tutorial, we’ll learn about ScalaPy, its features, and its usage. We’ll also discuss installing it and developing a basic program using it. Finally, we’ll see how to execute the ScalaPy program we wrote.

2. What Is ScalaPy?

ScalaPy is a Scala library developed by Shadaj Laddad. It lets us use any Python libraries from a Scala program. This enables us to bring together the features of both Scala and Python.

Python has a large number of Machine Learning and AI libraries, while Scala stands strong when it comes to big data processing. Therefore, both Scala and Python are focused on the same goal of enabling Data Science to be of great use to the community. ScalaPy was essentially brought into the world to make the implementation of data science applications easier and simultaneously make Scala more powerful. However, we can also use it to solve problems in many different domains.

2.1. Features

ScalaPy has a number of key features:

• It lets us use any Python libraries from a Scala program as if they were regular Scala APIs.
• It supports both Scala running under JVM as well as ahead-of-time compilation through Scala native-to-native code.
• It has a seamless interoperability layer provided by the CPython interpreter.
• We get a fully typesafe interface where we can interact with traditional Python libraries from the Scala program.
• In addition, ScalaPy supports creating type definitions to enable type-safe interactions with Python libraries.
• It’s easy to set up.
• It can do code auto-completion.

2.2. How Does It Work?

ScalaPy works by embedding a Python interpreter inside a Scala application. By importing the library dependencies for python-native-libs in plugins.sbt file, we can access the python libraries through Scala programs.

Internally, it communicates with a CPython interpreter. CPython is the default and most widely used implementation of the Python language. This means we’re actually using the regular off-the-shelf Python implementation within ScalaPy. As a result, we can import and use the Python libraries into the existing Scala code. Thus, all the libraries that work in Python can also work in ScalaPy. Therefore, we can get access to the vast ecosystem of Python libraries available.

Since it embeds a Python interpreter inside the Scala program, we don’t need to change any application or deployment setting. As a result, ScalaPy provides us with an identical interface to Scala Native or JVM.

Internally, ScalaPy also provides an intermediatory layer called “PyValue” which wraps a Python value to a low-level value. This also works the reverse way too. This means the low-level Scala values are also converted to Python values whenever needed.

Additionally, we can refer to a detailed description of ScalaPy features and its interoperability over Scala Programs.

3. Getting Started with ScalaPy

In this section, we’ll look at how to install ScalaPy and build a program that uses it.

3.1. Installation

Let’s understand the configurations required to run ScalaPy. We’ll see the steps for both JVM and Scala Native.

3.1.1. Installing ScalaPy on the JVM

ScalaPy is relatively easy to install. To start with, we need to add ScalaPy dependency to our build.sbt:

libraryDependencies += "dev.scalapy" %% "scalapy-core" % "0.5.3"

Then, we can set up ScalaPy with an SBT project by adding python-native-libs to project/plugins.sbt for both JVM and Scala Native:

// python-native-libs
libraryDependencies += "ai.kien" %% "python-native-libs" % "0.2.4"

Subsequently, for JVM, we add the below code in build.sbt:

fork := true
import ai.kien.python.Python
lazy val python = Python("<optional-path-to-a-python-interpreter-executable>")
lazy val javaOpts = python.scalapyProperties.get.map {
  case (k, v) => s"""-D$k=$v"""
}.toSeq
javaOptions ++= javaOpts

3.1.2. Installing ScalaPy on Scala Native

If we want to use Scala Native, we must first create a Scala Native 0.4.0-M2 project and then link the Python interpreter by adding the following configurations to our build.sbt.

Similar to configuring JVM, we need to add ScalaPy dependency to our build.sbt:

libraryDependencies += "dev.scalapy" %% "scalapy-core" % "0.5.3"

Next, to add python-native-libs, we need to add the same line as shown above for JVM. Finally, we can add the following code to our build.sbt:

import ai.kien.python.Python
lazy val python = Python("<optional-path-to-a-python-interpreter-executable>")
lazy val pythonLdFlags = python.ldflags.get
nativeLinkingOptions ++= pythonLdFlags

For more help on installation issues, we can make use of the ScalaPy community forum as well as the ScalaPy installation docs.

3.2. Python Support for ScalaPy

ScalaPy officially supports Python 3.{7, 8, 9}. However, if we want to use any other version of Python, we should either define the environment variable SCALAPY_PYTHON_LIBRARY:

python --version
# Python 3.9.9
export SCALAPY_PYTHON_LIBRARY=python3.9
sbt run

or we can set the system property scalapy.python.library:

sbt -Dscalapy.python.library=python3.9 run

However, we should note that the environment variable takes precedence over the system property.

3.3. Example

Let’s understand ScalaPy with a basic example. Let’s create a List in Scala and calculate its length using Python functions:

val listLengthPython = py.Dynamic.global.len(List(10, 20, 30).toPythonProxy)
val valueInScala = listLengthPython.as[Int]
println("Length of python list is: "+valueInScala)

We first created a Scala List and then used the toPythonProxy to convert it to a Python list. We can access any member of the global scope in Python using the py.Dynamic.global. Thus, we’ve used Python’s len() function to calculate the length of the list.

As a result, we get the Python integer value in the form of listLengthPython.

Finally, we can use the as() method to get the equivalent Scala value valueInScala out of a Python variable of any type.

Similarly, we can create a Scala Map and convert it to a Python dictionary:

val dictPython = py.Dynamic.global.dict(Map("India" -> "New Delhi", "Germany" -> "Berlin"))
val cap = dictPython.get("India")

println("Capital of India is: " + cap)

Here, we’ve used the get() method to get a Scala value from the Python dictionary. If the key isn’t present, the get() method returns a default value None.

3.4. Working With a Python Library

We can use py.module() method to import a specific module of a Python library. It returns an object of the module that we can use to access the methods provided by that module:

val math = py.module("math")

val squareRoot = math.sqrt(25)
// squareRoot = 5

3.5. Execution

We can compile the code using the SBT command compile. Then, we can execute the program using the run command in the SBT shell:

sbt run

Here’s the output of the execution of the above code examples:

Otherwise, we can use the nativeLink command to create a native executable for this project:

sbt nativeLink

This will create an executable file under the target/scala-<version> directory. We can then run this file as an executable, which will then print the text to the console.

Here’s more on building native applications in Scala using Scala Native.

4. Summary

In this article, we got an overview of ScalaPy, its features, and its usage. We also discussed the installation and learned how to write and execute a basic program that uses ScalaPy.

As always, the code for these examples is available over on GitHub.