How to Host Your Private GitHub Package on Streamlit Cloud
Written on
Introduction to Streamlit Applications
Streamlit is a powerful open-source Python library that simplifies the creation of interactive web applications tailored for data science and machine learning projects. It enables users to swiftly develop and deploy data-centric applications that can be shared through platforms like Streamlit Cloud, all while utilizing an easy-to-understand Python API.
Despite its user-friendliness, deploying a private package to Streamlit Cloud presents some challenges.
The Objective
Imagine we aim to create a Streamlit application that generates various random values, such as random integers, upon request. A prototype of such an application could resemble the following:
This Streamlit application is notably straightforward and can be implemented with fewer than 30 lines of code:
# main_page.py
import streamlit as st
from streamlit_random_generator.functions import random_int
from extra_streamlit_tools.utils import init_session_keys
init_session_keys({"randint_upper_limit": 10})
st.title("Generate a random number")
st.number_input(
"Max of random number",
on_change=random_int,
args=(st.session_state["randint_upper_limit"],),
key="randint_upper_limit",
min_value=0,
)
st.button(
"Generate",
on_click=random_int,
args=(st.session_state["randint_upper_limit"],),
)
if "generated_number" in st.session_state:
st.write(st.session_state["generated_number"])
# functions.py
import streamlit as st
import numpy as np
def random_int(x: int):
st.session_state["generated_number"] = np.random.randint(0, x + 1)
To ensure that this application is accessible to everyone, we want to host it online. However, sharing the source code may not always be desirable. To maintain confidentiality, we can store the code in a private GitHub repository.
The Challenge
Streamlit Cloud permits the installation of dependencies solely through a requirements.txt file. An example of a typical requirements.txt could include:
streamlit==1.19.0
extra-streamlit-tools==0.1.0
pandas==1.5.3
But what happens if you've developed your own Python package using a setup.py or pyproject.toml? If this package is published on a library host like PyPI, you can easily include it in your requirements file. However, many users keep their packages on GitHub, where installation is nearly as simple as from PyPI.
To install a package hosted in a GitHub repository, you can prefix the repository link with git+ in your requirements.txt:
streamlit==1.19.0
extra-streamlit-tools==0.1.0
pandas==1.5.3
This approach works seamlessly for public repositories. However, if your repository is private, like our random number generator application, Streamlit will produce an Error 128 during installation due to authentication issues.
The Solution
github_token = ""
Then, incorporate the following code snippet at the beginning of your main_page.py:
import subprocess
import sys
import time
import streamlit as st
try:
import streamlit_random_generator
except ModuleNotFoundError as e:
sleep_time = 30
dependency_warning = st.warning(
f"Installing dependencies, this takes {sleep_time} seconds.")
subprocess.Popen(
[
],
shell=True,
)
time.sleep(sleep_time)
dependency_warning.empty()
In this setup, during the initial loading of the page, the attempt to import streamlit_random_generator will fail, triggering a ModuleNotFoundError. Consequently, the user will see a message indicating that dependencies are being installed, which will take 30 seconds. This duration can be adjusted based on the number of dependencies.
While the installation occurs, the subprocess module will install your package and its dependencies from the private GitHub repository, using the GitHub token stored in Streamlit secrets. Upon the next reload of main_page.py, the import will succeed, avoiding further installations.
For access to the GitHub repository containing the source code, click [here](#).
Conclusion
In this guide, we've explored how to leverage Streamlit Cloud while preserving the privacy of your package code on GitHub. Should you face similar authentication challenges when installing a private GitHub package, these strategies could also apply.
To further your knowledge of Python, feel free to check out my other articles!
Resources
More content available at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord. If you're interested in scaling your software startup, explore Circuit.
Chapter 1: Deploying Streamlit Apps
The first video titled "Deploy Streamlit Apps For Free on Community Cloud" explains how to effectively deploy your Streamlit applications without cost.
Chapter 2: Streamlit Cloud Deployment
The second video, "How to deploy a Streamlit app to Streamlit Cloud," provides a step-by-step guide on deploying your applications to Streamlit Cloud.