Upstream Impact | 2i2c

Supporting JupyterHub admins on workshop hubs with shared passwords

Mon, 27 Apr 2026 00:00:00 +0000

To facilitate communities using JupyterHub for a workshop, we introduced the idea of a ‘shared password’ based authentication a few years ago. This lets communities set a single global password that is handed out to all workshop attendees (instead of collecting email addresses or GitHub usernames before the workshop starts). Users can login with their email and this shared global password, and get going immediately. We worked with OpenScapes (which runs a lot of workshops) in developing this method, and it’s been heavily used since among other communities!

However, until now this method required us to restrict admin access for all users, preventing users from using admin features like group management, accessing the JupyterHub Admin dashboard (which lets you see which other users are logged in), and using the shared directory.

To mitigate these, we contributed the SharedPasswordAuthenticator upstream to the JupyterHub organization, allowing hub admins to set set two passwords instead of one. Hub admins can now generate a special password that can be distributed just to admins, and that gives users these extra capabilities without giving up the simplicity of using a single password for all workshop participants!

By contributing this upstream, we made sure this feature is available for everyone to use, rather than just for communities served by 2i2c. We’re rolling this out to our communities now, and many communities have already benefitted from this!

Acknowledgements #

To MinRK for code review and merge of the pull requests!
The OpenScapes community for collaborating with us on building features that benefit themselves and everyone else!
Support from our member communities gives us the capacity to invest in upstream open source engagement and build relationships like this

Report from the Jupyter and AI community meetup

Thu, 23 Apr 2026 00:00:00 +0000

A small group of Jupyter community members recently met in Seattle to demo Jupyter workflows augmented with AI tools and discuss what’s next. This is a quick report-out about what stood out.

The conversations focused on what agentic workflows could unlock for researchers and students - with broad acknowledgement that deployment, security, authorization, and ethics still have a lot to be worked out.

Here are a few highlights from the demos.

Jupyter AI and the Agent Client Protocol #

Jupyter AI v3 is adding support for the Agent Client Protocol (ACP), enabling multi-agent collaboration within JupyterLab. We saw how a single request could be split across multiple specialized agents based on each one’s skills, similar to this demo from JupyterCon 2025.

Some things still to figure out:

No tool isolation yet - agents aren’t scoped by user role or security group.
Manual configuration required - agents running locally need to be registered by hand.
A future goal is for JupyterLab to auto-detect locally installed agents, or for LLM providers to contribute integrations directly.

The Jupyter AI team also shared a new “Personas” abstraction that lets you define a skill-specific agent in just a few lines of Python. Personas open the door to domain-specific agents or agents with crafted “personalities”. They are written in a few lines of Python, though we discussed the possibility of using markdown instead. This seems to be evolving rapidly!

Jupyter AI v3.0.0 has since been released.

Notebook CLI #

nb-cli is a Rust-based tool for interacting with notebooks from the command line - changing cells, running them, checking errors, and rendering the notebook in the console. It can also connect to remote JupyterLab instances, making it useful for headless workflows like CI pipelines or batch grading.

Sidebar comments and Real-time collaboration #

A demo showed Google Docs-style side-panel comments for .ipynb files. Previous collaboration efforts used Conflict-free Replicated Data Types (CRDTs) for multi-cursor editing, but the team felt that approach may not have hit the mark. Side-panel comments felt more natural, though how this extends to .py files is still an open question.

Opportunities for 2i2c #

The meeting helped us understand the role we could play in this space. We think there’s opportunity to help facilitate the use of agents on shared infrastructure, particularly around the upstream control planes for deployment, administration, and authorization. We can also help facilitate community conversations about the ethics and impact of AI tools on open science.

Acknowledgements #

The Jupyter Foundation for supporting the event.
Zach Sailer (Apple), for organizing this meetup.
Chris Holdgraf for editing and adapting the original content for this post.

Report from the Jupyter Security Working Group security tooling sprint

Wed, 08 Apr 2026 00:00:00 +0000

The Jupyter Security Working Group recently held a Security Tooling Sprint. It was a timely event given the recent spate of software supply chain attacks across the tech world.

The sprint covered two main areas:

Governance and strategy — conversations about responsibility and accountability in the face of AI, with emphasis on ensuring humans are ultimately responsible for code committed to Jupyter subprojects. The group also discussed how security could benefit from working group members regularly attending subproject meetings like the JupyterHub Collaboration Cafes.
Automation and tools — the group evaluated several tools for improving security posture across the Jupyter ecosystem. Here are a few that stood out:
- Semgrep as an alternative vulnerability scanner to CodeQL
- Grype, Checkov, and Kubescape for cloud infrastructure misconfiguration checks
- Schemathesis and restler-fuzzer for API fuzz testing

One challenge we discussed was how blindly running security scanning tools generates many false positives. There’s real effort needed to tune these tools for each project’s edge cases before they’re useful in automation. On a related note, we discussed the increase in AI-generated (or AI-assisted) vulnerability and security reports, and the challenges associated with sifting through all of those pieces of information.

Acknowledgements #

Thanks to the jupyter security working group for providing leadership and organizing, in particular Joe Lucas!
Thanks to the Jupyter Foundation for funding community meetings like these.

Combining multiple repos into one site at jupyterbook.org

Mon, 06 Apr 2026 00:00:00 +0000

As part of an initiative to improve jupyterbook.org’s documentation, we refactored the site so that multiple repositories are served under one domain. We wrote up the details on the Jupyter Book blog.

Read the full post here:

How we combine multiple repositories into one website at jupyterbook.org.

Along the way we made several upstream improvements to the MyST Engine and the MyST theme:

parts: support for extends: and URL support as well so we can share a footer / navbar configuration across repositories
internal_domains option so that links to another repository’s content could still be treated as internal
Less aggressive citation parsing so that text like @githubhandle weren’t parsed as a citation
Several mobile and UX fixes (that’s one, there were many others!)

These all felt particularly relevant for documentation that our member communities manage, where you have content split across multiple repositories but served at a single domain.

Acknowledgements #

Thanks to Project Pythia and EarthScope for collaboration and feedback that helped shape this work. And thanks to our member communities whose memberships fund upstream contributions like these.

Jenny Wong joins the JupyterHub team

Mon, 09 Feb 2026 00:00:00 +0000

We’re excited to share that Jenny Wong has been invited to join the JupyterHub team as a contributor and maintainer.

Jenny’s contributions to nbgitpuller and grafana-dashboards, along with her active participation in project meetings and community planning, earned her this recognition from the JupyterHub community. Being invited to a project team means that existing team members have recognized a pattern of high-quality contributions and trust in a person’s ability to steward the project.

We’re particularly excited about this because our mission isn’t just about deploying infrastructure - it’s about being good citizens in the open source communities we depend on. We invest in upstream contributions, participate in community governance, and aim to build the kind of relationships that strengthen the whole ecosystem. When our team members are welcomed into upstream project teams, it’s a signal that we’re doing this well.

Learn more #

JupyterHub team compass - how the team is structured and how new members are added
JupyterHub proposal to add Jenny to the team
nbgitpuller and grafana-dashboards - projects Jenny contributes to

Acknowledgements #

The JupyterHub community for fostering an open and welcoming project culture
Support from our member communities gives us the capacity to invest in upstream open source engagement and build relationships like this

Introducing Jupyter Book 2 at FOSDEM 2026

Fri, 06 Feb 2026 00:00:00 +0000

Our teammate Angus Hollands gave a talk at FOSDEM 2026: Introducing Jupyter Book 2.

The talk shares why the Jupyter Book 2 and MyST stack was rebuilt, and how it supports open, reusable computational publishing workflows.

Slide from the FOSDEM 2026 session introducing Jupyter Book 2.

Learn more #

Acknowledgements #

Thanks to Project Pythia and to BIDS for supporting deeper foundational contributions like this in Jupyter Book.
Thanks to the Jupyter Book team for helping out with slide creation and doing all the work Angus spoke about here!

New Jupyter Book / MyST stack release (Jan 2026)

Thu, 29 Jan 2026 00:00:00 +0000

The MyST/Jupyter Book stack shipped new releases this week. Release pages:

Where we contributed #

We’ve spent extra time lately trying to fix bugs and generally improve stability, reliability, and UX papercuts. Here are some of the things we focused on:

Extra review time to get in concurrent executions in mystmd
Fixing broken edit URL logic so our community “edit buttons” worked again
Standardized link styles so users know what to expect from links
Added extra hover metadata for github issues and PRs for communities that often link to their GitHub issues
Made all of these releases and wrote up the release notes!

Most of our contributions were foundational in nature - we fixed a bunch of bugs, did review on the PRs of others, and managed the release process itself. Check out the changelogs for more details!

Acknowledgements #

Thanks to our member communities - their memberships cover the cost of foundational upstream contributions to projects like these.

Particular thanks to Project Pythia which currently supports much of our upstream contributions in Jupyter Book.

Yuvi on scaling maintainer intuition to facilitate PR review with PR triage boards

Wed, 19 Nov 2025 00:00:00 +0000

Yuvi has a recent post on the Jupyter blog on how his “maintainer intuition” about reviewable pull requests grew into the open-source pr-triage-board-bot, a reusable workflow that keeps GitHub Project boards curated for the JupyterHub, JupyterLab, and GeoJupyter communities. Foundational contributions are rellay important to 2i2c. This is a great example of building clever technical systems that help maintainers prioritize the social work of facilitating contributions to keep ecosystems healthy.

The JupyterHub PR Triage board.

Our favorite quote shares the vibe and cultural principles that drive this effort:

If the author of the PR is a newish contributor, I want to encourage them to stick around by being responsive to their gift. All PRs are gifts that we may or may not choose to accept, but should do so with grace.
Yuvi

Yuvi frames the problem here:

Reviewing PRs is a critical way that maintainers keep an open source project moving forward, but identifying PRs that can productively be merged is hard.

And notes that human scalability is often a big bottleneck:

One key bottleneck we identified in the process was Step 2. In particular, I was relying on my maintainer intuition to pick a single PR that I believe can be merged, so others in the team can do review work. I started exploring what this intuition is, and if it can be scaled.

Here’s his list of “intuitions” that he uses to choose PRs to work on:

PRs that aren’t too big, and are a reasonable size that can be merged within a 2 week window

CI tests passing, so at least our automated checks haven’t caught any issues with it Features or bug fixes that I believe add value to the project and move us in the right direction towards being able to support our users as they need (this is the hardest!)

If the author of the PR is a newish contributor, as I want to encourage them to stick around by being responsive to their gift. All PRs are gifts that we may or may not choose to accept, but should do so with grace.

How long ago the PR was opened. There is such a big difference between a response to your PR 2 days after you make it vs 2 months vs 2 years. I prioritized newer PRs.

What kind of contribution is it primarily? Different engineers on our team have different skillsets (JS, Python, etc) and I wanted to match the PR to what the engineer preferred code reviewing.

And the board essentially tries to capture many of these intuitions by signal-boosting them in one place:

we can roughly say ‘Pick a PR that looks good to you from the top of the “First Time Contributor” or “Seasoned Contributor” list’, and that relieves me from being the bottleneck quite a bit.

Read more in the original article: Scaling “Maintainer Intuition” with Pull Request Triage Boards.

Acknowledgements #

Project Jupyter for trusting us to incubate and now donate the bot code to the broader ecosystem.
JupyterHub and GeoJupyter contributors who tested the triage views and fed real maintainer workflows back into the design.
Jason Grout, Raniere Silva, and Matt Fisher for spotting the experiment early and helping it land across multiple orgs.

Creating a re-usable redirect generator for Jupyter Book 1 migrations

Wed, 12 Nov 2025 00:00:00 +0000

When migrating documentation from Jupyter Book 1 to Jupyter Book 2, URL structures change dramatically and break external links. We spent some time createing a re-usable tool to solve this problem across multiple projects.

You can check out the tool below:

jupyter-book/jb1-redirect-generator

It’s designed to be run in a self-contained way by putting the dependencies in script metadata at the top. This means you can run it like this:

uv run https://raw.githubusercontent.com/jupyter-book/jb1-redirect-generator/main/generate_redirects.py

This let’s you generate redirects from JB1 -> JB2 URL structures and dump them in a _build/html folder with your JB2 built pages.

We tested this out by converting the Jupyter Governance docs to Jupyter Book 2 and running it there. You can find a noxfile that runs these commands here:

jupyter/governance/blob/bcdae30efdecbe75bc4751ef1fe1e602fe82ee10/noxfile.py#L25-L37

And its use in a GitHub Workflow here:

jupyter/governance/blob/bcdae30efdecbe75bc4751ef1fe1e602fe82ee10/.github/workflows/deploy.yml#L39-L44

Learn more #

Refactoring Jupyter Book 2 documentation ahead of a major release

Sat, 01 Nov 2025 00:00:00 +0000

Documentation is what turns open source code into products that people actually want to use. We recently spent a few days refactoring the Jupyter Book documentation to prepare for the upcoming Jupyter Book 2 release, and we’re excited about how much clearer the docs have become!

What we did #

We restructured the docs using the Diataxis framework to better organize content by user type and task:

Reorganized into clear topic areas with landing pages for easier navigation
Added missing content like the feature voting table and contributing guides
Created upgrade guidance to help users understand the relationship between Jupyter Book 2 and MyST

This work helps users find what they need faster and gives the project a stronger foundation to build on going forward.

Why we’re excited about it #

Better documentation reduces maintainer burden by helping users answer their own questions, and it makes the project more welcoming and useful to new contributors. We hope this makes Jupyter Book more accessible to everyone and lays a good foundation for the new release!

We’re also excited because so many others helped provide edits and comments!

Learn more #

Acknowledgements #

Thanks to @rlanzafame, @FreekPols, and @bsipocz for their helpful reviews, edits, and feedback on the PR!
Project Pythia, CryoCloud, and the Berkeley educational projects are our primary member communities using MyST and Jupyter Book. Their support covers the cost of these kinds of foundational contributions.

Fixing the mybinder.org usage analytics archive

Tue, 14 Oct 2025 00:00:00 +0000

The analytics archive at archive.analytics.mybinder.org powers the mybinder.org usage dashboards and provides a daily-published dataset that researchers and communities use to understand how Binder is being used across different domains and scientific communities.

While updating our quarterly Binder impact report, we discovered the archive index page had stopped updating. The analytics publisher was writing index files to temporary storage before uploading to Google Cloud Storage, but for some reason the upload step stopped working. We deployed a fix that eliminates the temporary files entirely - the code now generates the HTML index as a string in memory and uploads directly.

The mybinder.org analytics archive shows a list of daily usage reports that anybody can download.

Fortunately, we didn’t lose any data! Thanks to some smart design decisions, the daily analytics files were being collected properly the entire time, only the index page listing them was broken. You can find the full archive here.

Learn more #

Pull request with the fix
mybinder.org usage dashboards
The binder-data/ repository is where we aggregate and publish archive data to be more accessible.
Our quarterly impact report from mybinder.org

Acknowledgements #

Thanks to the JupyterHub community for their collaboration on mybinder.org infrastructure

Impact report from 2i2c's Binder federation instance

Tue, 14 Oct 2025 00:00:00 +0000

The mybinder.org service provides reproducible and interactive computational environments for the open science community. It is financially supported by a federation of BinderHubs. 2i2c is part of the team that manages mybinder.org, and we’re committed to supporting this critical infrastructure for open science and reproducibility.

We developed a more cost-efficient process for deploying BinderHub on a single VM and are now running a BinderHub at 2i2c.mybinder.org. This post highlights the impact we’ve had through our support of the mybinder.org federation, we’ll update it periodically.

Usage over time #

Here are weekly launches on mybinder.org. Launches from 2i2c.mybinder.org are in red. Source: mybinder analytics dashboard.

In Q1 of 2025, 2i2c.mybinder.org launched 417,048 reproducible sessions. In this time, mybinder.org was primarily driven by our new Hetzner node as we worked with GESIS to stabilize their own BinderHub instance.

In Q2 of 2025, 2i2c.mybinder.org launched 249,750 reproducible sessions. In this time, we worked with GESIS to deploy their BinderHub instance on the same Hetzner node setup, which let us distribute more of Binder’s load onto them.

In Q3 of 2025, 2i2c.mybinder.org launched 118,083 reproducible sessions. We experienced a typical summer dip in sessions, had to fix the analytics dashboard as well as fix a TCP scanning abuse case that briefly brought down the 2i2c node.

Where we’ve made improvements #

As part of this effort, we’ve made several improvements to the Binder and JupyterHub ecosystem. Here are a few links where you can read more:

Deploying BinderHub on a single VM with k3s - Our approach to making BinderHub deployment cheaper and simpler
Hetzner cloud infrastructure experience - Cost-effective cloud hosting for mybinder.org
Combating TCP scanning abuse on mybinder.org - Developing anti-abuse tools to prevent an abuse use-case.
Improving Binder’s usage dashboard - This helps us create posts like these and is useful to others as well.
Integrating BinderHub with JupyterHub - Working with GESIS to bring Binder’s dynamic image building capabilities to persistent JupyterHubs, empowering users to manage their own environments

Acknowledgements #

This work is made possible by:

GESIS for their continued support as mybinder.org federation members
The JupyterHub community for collaboration and support
Our member communities whose fees support this work

Combating tcp scanning on mybinder.org with the tcpflowkiller

Wed, 08 Oct 2025 00:00:00 +0000

We’ve deployed a new tool to mybinder.org that automatically detects and stops port scanning activity, helping us maintain service reliability while being responsible citizens of the internet.

Port scanning is a common part of network-based exploits, and many server hosts prohibit this activity (including Hetzner, where the 2i2c mybinder.org infrastructure lives). We developed a little tool called tcpflowkiller as part of the cryptnono project (our anti-abuse set of tools for hosted JupyterHub and Binder infrastructure) to automatically kill processes that exhibit port scanning behavior. This reduces the likelihood of triggering our server host’s abuse policies and helps keep mybinder.org running reliably.

Why this matters #

As providers of public compute, it’s our responsibility to make sure people can’t use our infrastructure to abuse others. This is part of being responsible citizens of the internet. It also saves us time in dealing with outages because cloud providers (understandably) block access when they suspect there is abuse.

Hetzner and similar hosts have many benefits (including significant cost savings), and tools like tcpflowkiller help keep hubs and binders running smoothly on such hosts, which have different abuse policies than the big commercial cloud providers.

AWS and other cloud providers have proprietary ways to combat abuse (like AWS GuardDuty). We could have spent our time investing in developing rules there. Instead, contributing to cryptnono helps provide the same set of features in a cloud-agnostic way, in line with our principles of supporting open infrastructure that gives communities control over their infrastructure.

This tool has now been deployed to mybinder.org, and we’ll monitor its effectiveness over time. We may roll this out to 2i2c public BinderHubs in the future based on patterns we observe.

Learn more #

Acknowledgements #

Thanks to GESIS for their continued support of mybinder.org and to Raniere Silva for collaborating on this deployment with us.

TIL: GitHub Action secrets are only available from non-forked repositories

Wed, 08 Oct 2025 00:00:00 +0000

If you’ve worked with GitHub Actions in open source projects, you might encounter a hard-to-debug error where repository secrets are simply empty. That’s probably because the PR is from a forked repository! Here’s a little learning we had after losing a bunch of time figuring this out:

Our PR from a fork was using empty strings for repository secrets #

github-activity is a tool we help maintain for generating changelogs from a wider variety of contributions than GitHub’s defaults. We needed to set a secret to raise the API rate limits for the tool’s tests. These tests pull data from repositories outside of the github-activity repository itself, which quickly hit GitHub’s rate limits without authentication.

The problem seemed straightforward at first: we added the secret, but the workflow was acting as if the secret didn’t exist at all. After updating the code to explicitly check for empty strings, we discovered the authentication token was actually an empty string, even though it had been set.

After some debugging, we uncovered the root cause: the PR was opened from a fork of github-activity. GitHub intentionally makes secrets appear as empty strings when a PR originates from a forked repository. This is a security measure to prevent unauthorized access to sensitive credentials.

A quick fix: re-open the PR from the base repository #

The immediate fix was simple: re-open the PR from a branch in the base repository rather than from the fork. This worked perfectly, but it’s not a sustainable solution for open source projects that rely on community contributions from forks. We don’t want to create a dynamic where maintainers have different PR workflows because they’re operating on the base repository.

How use secrets with forked repositories in a safe-ish way #

If you need to make secrets available to PRs from forks, there are a few approaches we learned about, each with security trade-offs:

The `pull_request_target` workflow #

GitHub provides a pull_request_target workflow that can access secrets even when triggered by forked PRs. In this case, GitHub will always run the test suite on main, instead of any changes your PR introduces.

Why this is dangerous: malicious actors could add code to a PR that exfiltrates your secrets (for example, Python code that prints os.environ["MY_SECRET"]).

As a result, only use secrets that you’re OK with being public. In this case, we generated a read-only token with restricted permissions. However, this is still kinda risky so use at your own peril.

If your repository workflows require a secret that absolutely cannot be public (e.g., a publishing key for a package repository), try a method like the following:

Using GitHub Environments for granular control #

A safer approach is to use GitHub Environments, which let you restrict which secrets are available to specific jobs. This way, you can ensure that only non-critical secrets (like those needed for testing) are available to jobs that run on forked PRs, while keeping sensitive secrets (like PyPI publishing tokens) restricted to trusted contexts.

This is the approach we implemented in github-activity, and it provides a good balance between security and community contribution workflows. We created a separate environment for publishing to PyPI so that its secret is never available to the job that runs with pull_request_target.

We hope this saves you time! #

Hopefully this learning is useful to others who run into the same confusing behavior. We’ve added a few improvements to github-activity to more reliably check for empty strings to surface this kind of condition, but knowing the basic behavior of GitHub environments and forked PRs is even better.

Learn more #

Updates from Chris' position on the Jupyter Executive Council and Foundation Board

Tue, 23 Sep 2025 00:00:00 +0000

This is a running blog post for Chris to share out community updates from his time on the Jupyter Executive Council. See context for this page.

Updates for the Jupyter community from Chris #

Chris aims to write monthly updates for Jupyter’s community in the Jupyter Community Forum to share his perspective on what the council is up to. We’ll update this index post as new posts are available.

Blog posts about Chris’ experience on the executive council #

Chris blogs now and then to share his thoughts, important announcements, and provide major updates from his perspective on the executive council. Here are the posts that have come out of this effort.

Your Ideas, Our Support: Jupyter Community Call For Funding Proposals
Why open source foundations try to fund systems, not development - May 31, 2025
Jupyter can align the needs of its community and its foundation by enabling contribution - Mar 22, 2025
The relationship between the Jupyter Executive Council, Software Steering Council, and Foundation - Mar 02, 2025
Ways the Jupyter Foundation could support open source projects - Feb 26, 2025
Running for the Jupyter Executive Council - Jan 14, 2025

2i2c’s commitment to supporting open source communities #

In early 2025, Chris was elected to the Jupyter Executive Council. We’re tracking these efforts because providing leadership and community support to open source projects is a key Foundational contribution we wish to make for open source projects. 2i2c makes time for team members to contribute upstream in strategic ways like this as part of our commitment to open practices. However, this kind of work is very difficult to track! This blog post is an attempt at putting these efforts into one place for us to track and for others to discover.

Acknowledgements #

2i2c’s member communities support Foundation open source contributions like this with their membership fees.
The Navigation Fund provides strategic funding support for 2i2c which covers major open source leadership contributions like this.

Sharing JupyterHub's vision for more flexible application deployment at the doepy talk series.

Wed, 03 Sep 2025 00:00:00 +0000

Our Technical Lead Yuvi Panda recently gave a talk at the doepy meetup about JupyterHub’s interest in moving beyond the “single-user notebook application” and towards a more flexible approach to enabling administrators to deploy many different types of applications and environments.

Check out a video of the talk here:

This is an important step for the JupyterHub project in order to support the many different kinds of workflows that data scientists need to use in their work. We hope that this generates more interest in the JupyterHub project and gives us useful feedback to guide the team’s understanding of this direction.

Learn more #

Acknowledgements #

The doepy team for inviting Yuvi to give this talk.
The JupyterHub team for working with us on this strategy.
2i2c’s network of member communities whose fees support our Foundational open source engagement.

Overhauling repo2docker's documentation

Fri, 01 Aug 2025 00:00:00 +0000

Documentation is incredibly important for open source projects to communicate their value and show users how to make the most of their tools. However, it’s one of those things that often gets de-prioritized with all of the other work that needs to happen in a project.

We are heavy users of the repo2docker project in 2i2c’s service. It allows you to dynamically build an environment image that can by hosted in a cloud service like JupyterHub. It’s the underlying tool used by BinderHub, and is the focus of recent work on enabling dynamic image building in a JupyterHub (more on that to come).

As part of this work, we decided to do a small overhaul of repo2docker’s documentation, with the goal of making it easier to discover, navigate, and learn from. Here’s how the landing page looks now!

The new repo2docker landing page now looks more like an actual landing page!

We hope this makes repo2docker a more useful tool for everybody, and also gives us more confidence pointing our communities to the repo2docker documentation in their community workflows.

Acknowledgements #

Thanks to the NASA VEDA project for providing funding and collaboration for this work.
Thanks to the JupyterHub community for collaboration and review of this work.

2i2c's submissions to JupyterCon 2025

Sat, 19 Jul 2025 00:00:00 +0000

Update: The talks are now live! See this blog post with links to our videos.

We were excited to hear that JupyterCon is happening again in 2025. The Call for Proposals just wrapped up, and our team was involved in preparing and submitting several directly from 2i2c as well as from the ecosystem in general.

Enjoy brief summaries of the proposals we contributed to below. Let us know if you have ideas for future talks you want to hear from us?

Note: Many of these submissions were written in collaboration with others in the open-source projects we participate in. Particularly, in JupyterHub and JupyterBook.

JupyterHub: A multi-user server for Jupyter notebooks #

This is how JupyterHub was described when it was announced in 2015, 10 years ago. The focus was on bringing Jupyter Notebooks to multiple users on shared infrastructure. The Jupyter Notebooks focus was so strong that Jupyter was even in the name of the project! Fast forward 10 years, & this is still the most common perception of JupyterHub.

However, this has not been true for a long time now. Instead of setting up 5 different kinds of infrastructure to support your users based on what kind of interface they like to use ( JupyterLab, RStudio, Linux Desktop tools like QGIS or Napari, Visual Studio Code, full ssh (!?), etc) for their interactive computing, you can set up a JupyterHub that supports all of those! Meet your users where they are, rather than force them to conform to using only a specific set of tools.

Come to this talk to:

See cool demos of various popular applications running on JupyterHub seamlessly
Understand the security model of JupyterHub & how that enables these cool demos
Learn how you can set up your own application to run in JupyterHub
Influence the future of how JupyterHub is marketed

Cloudy with a Chance of Savings: Per-User Usage and Cost Monitoring for JupyterHubs in the Cloud #

Cloud cost monitoring is moving beyond just preventing runaway cost explosions – it’s about empowering JupyterHub administrators with the guardrails they need to run efficient, transparent, and sustainable infrastructures. A cloud cost bill can show a broad view of services and machines provisioned, but how can we provide granular insights into each user and the value they are deriving from the hub on an application level?

We’ve developed several open-source components towards answering this question:

Metric Collection – Prometheus collects resource usage metrics (including CPU, memory, and storage) from individual user pods via standard and custom exporters.
Cost Estimation – Usage is correlated with AWS cost data to estimate per-user costs.
Visualization – Grafana dashboards display rich, interactive views of usage and cost data, making it easy to monitor trends, identify high-cost workloads, and generate reports for funders and decision-makers.

This approach delivers cloud observability and cost transparency that can be reliably deployed using Kubernetes and integrated with Zero to JupyterHub distributions.

Controlling home directory costs (with user empathy) on the cloud with jupyterhub-home-nfs #

User home directories on JupyterHubs deployed in cloud providers has been a pain point for both end users and administrators. Administrators feel the pain of cloud costs for home directory storage (sometimes higher than compute!). No user wants to receive an email saying “well, this code you copy pasted downloaded 3TB of netcdf files into your home directory, and now we have used up our entire team’s cloud budget for the next 2 years” (true story).

The jupyterhub-home-nfs open source project is a JupyterHub native, cloud agnostic solution to these problems. Administrators can do per user limits, tune performance, report on usage and make cloud cost conscious choices around overprovisioning. It provides users empathetic guardrails to prevent them from overuse, rather than punitive gates that zap them after the fact.

In this short talk, we will:

Describe the core of the problem, and how it manifests for users and admins.
Review current solutions and their limitations
Introduce jupyterhub-home-nfs and how it moves the solution space forward
Demo how this looks like for an end user
Talk about future direction, and opportunities for collaboration

Rebuilding user trust in `mybinder.org` #

Have you clicked a mybinder.org link, waited for it to start and gave up after it took far too long to start? Or just failed?

Have you stopped using mybinder.org for your tutorials, repositories and presentations because you could no longer rely on it to work each time?

mybinder.org is open infrastructure run by incredible volunteers (who go above and beyond constantly) in the Jupyter community. Is ‘slow fade into unreliability’ just the fate of all openly run infrastructure?

But perhaps maybe, just maybe, you have tried doing that again recently, & noticed improvements! Launches are more reliable. Faster. The UI looks better. Maybe things are getting better?

This talk will go through the problems facing mybinder.org & what we are doing about it. Come to this talk to find out:

How is mybinder.org run?
What are the structural issues facing open infrastructure services like mybinder.org?
What sustainability experiments are we running to improve reliability and rebuild user trust?
Has reliability actually improved?
How can I help?

Can your JupyterHub handle your workload? Performance testing with `jupyterhub-simulator` #

Your JupyterHub is all set up, and you’re excited to use it for your workshop of 60 students. Or your class of 600 students. Or your research group of 5 people with complex workflows.

You feel your infrastructure should hold up fine, but do you know if your infrastructure will hold up fine? Is that just excitement, or is there a little bit of nervousness in there too? Wouldn’t it be nice to test and know for sure?

jupyterhub-simulator is an open source project that allows you to describe what you expect your users to be doing - starting servers, clicking on nbgitpuller links, running notebooks, etc. Once you have described it, you can then simulate any number of users doing that workflow simultaneously, and verify that your JupyterHub can handle that workload. If it can’t, tweak your infrastructure and try again until it does!

In this talk you will learn:

What is jupyterhub-simulator?
How can I describe my expected workflow?
How can I test if N users can do this workflow all simultaneously?
How can I visualize the performance of my infrastructure so I can tweak its configuration and try the simulation again?

Not just for notebooks: JupyterHub in 2025 #

JupyterHub: A multi-user server for Jupyter notebooks

Come to this talk to:

See cool demos of various popular applications running on JupyterHub seamlessly
Understand the security model of JupyterHub & how that enables these cool demos
Learn how you can set up your own application to run in JupyterHub
Influence the future of how JupyterHub is marketed

Introducing Jupyter Book 2.0 #

This is a community talk from the Jupyter Book team, detailing the principles behind the new MyST Document Engine and Jupyter Book 2’s upcoming release. We’ll share the text when the Jupyter Book team posts it publicly.

You don’t need to contribute more to open source, you need to go to therapy #

Open source communities can be incredible and irreplaceable sources of human connection in our lives, offering a unique kind of fulfillment hard to find elsewhere. This feeling of fulfillment and approval can, for some people, be a soothing balm in an otherwise rough life. However, it has the be part of a healthy, balanced ecosystem of different kinds of connections offering different kinds of fulfillment . If a significant chunk of fulfillment in your life comes from the open source work you do, unbalanced by other sources, that can quickly become unhealthy for both you and the community. Disagreements are more likely to become high stakes. Interactions can quickly become emotionally charged and filled with hard to interpret subtext. This can both burn you out, and drive away potential new community members.

This talk explores the why emotional regulation is a critical skill for participating in open source communities, and how therapy can be a tool for learning that skill. Come to learn:

What is emotional regulation?
Why should I care?
What is therapy and how can I access it?
Wait that’s not what I thought therapy was! Are you telling me Missy Armitage lied in Get Out?!
What negative effects do communities feel?

What 2i2c has learned while trying to build sustainable relationships with Jupyter’s community. #

2i2c is a non-profit organization that fosters co-creation and collaboration between science communities and open source communities. We are deeply embedded in an international network of research and education communities, as well as open source communities that underlie their infrastructure (particularly in the Jupyter ecosystem). Our technical infrastructure is built entirely with open source components that we contribute to, but do not control. This is a really hard problem to solve!

We’ve learned a lot in the first four years of our existence. This talk will describe how our organization approaches a healthy and productive open source relationship with the Jupyter (and broader scientific python) ecosystem. It’ll cover some of the major mistakes we’ve made, lessons learned, and where we think we’ve had impact. We aim to make this talk full of practical learnings that others can follow in building sustainable open science organizations that contribute to a healthy and vibrant open source ecosystem. Our goal will be to provide inspiration to others that are interested in building on top of open source projects like Jupyter, and want to do so in a way that is healthy and sustainable.

Open slides: Jupyter Book 2 and MyST at the UC Berkeley Data Science Education Summit

Tue, 01 Jul 2025 00:00:00 +0000

Chris gave a talk about Jupyter Book 2 and MyST at the UC Berkeley Data Science Education Program’s annual meeting. It covered the next direction for the Jupyter Book project, and its recent adoption of the MyST Document Engine for Jupyter Book 2.

You can view the full slide deck here.

Learn more #

Acknowledgements #

Thanks to Project Pythia for funding some of our work on the Jupyter Book and MyST ecosystem
Thanks to CloudBank for collaborating with us on adapting and deploying Jupyter Book for education and organizing this summit
Thanks to the Jupyter Book project for collaborating with us on these strategic efforts over the last years

Jupyter Book at the Scientific Python 2025 Developer Summit

Fri, 23 May 2025 00:00:00 +0000

Chris and Angus recently attended the Scientific Python 2025 Developer Summit on behalf of Jupyter Book, here’s a brief blog post about their experience written with the Jupyter Book team.

Chris is joining Project Jupyter's Executive Council

Mon, 10 Mar 2025 00:00:00 +0000

We are proud to announce that 2i2c’s Executive Director, Chris Holdgraf, was recently elected to Jupyter’s Executive Council. The 2i2c team discussed whether Chris should run for this position last year, and concluded that it was a way for our non-profit to both support Jupyter’s mission at a strategic level, and represent the interests of research and education communities in Jupyter’s direction. Chris wrote a blog post about his reasons for running with more information.

One of Chris’ goals is to be a transparent source of information about what the council is working on, where its priorities lie, and what are the major challenges it is trying to tackle. He’s written two blog posts that describe some of his experiences so far, at the links below:

We’re hopeful that this is a way for 2i2c to scale its impact and lean into its commitment to open technology. Chris intends to keep writing about his personal experience via his blog, and we’ll provide updates here for major developments that are relevant to 2i2c’s network of communities. We’re proud to have Chris in this role, and excited for his contributions to the Jupyter community!

Acknowledgements #

Strategic open source support like this is supported by a grant from The Navigation Fund and fees from our member organizations.

Simplifying and speeding up Binder builds with BuildKit

Mon, 03 Mar 2025 00:00:00 +0000

Chris and Yuvi recently wrote a blog post on the Jupyter blog about a recent experiment to significantly reduce the cost of running a node on the mybinder.org federation.

Acknowledgements #

Project Pythia provides support for some of our work with the Binder project.
JupyterHub for working with us to get this new node deployed for mybinder.org.

2i2c joins the mybinder.org federation with a cheaper and faster way to deploy Binderhub

Wed, 29 Jan 2025 00:00:00 +0000

If you’re interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the

Support Binder page for how you can help.

tl;dr: The 2i2c team is joining the mybinder.org federation with a single-node BinderHub instance at 2i2c.mybinder.org. It should be much cheaper to run than auto-scaling Kubernetes clusters, and might be a good way to support mybinder.org more sustainably. For questions or comments, join this Jupyter Zulip thread.

mybinder.org is a massive public service for creating and sharing reproducible computational environments. It is managed by the JupyterHub team and members of the mybinder.org federation. One challenge in running mybinder.org is identifying cloud credits or financial resources to support the cloud infrastructure that runs the service. Two years ago, Google stopped supporting mybinder.org federation with cloud credits, and last month the federation lost more capacity, leaving only GESIS and OVH as remaining federation members¹. This makes mybinder.org less reliable, slower, and generally less useful to the world.

The landscape of cloud infrastructure technology and services has changed considerably, and we think that there’s a way to deploy BinderHub instances with lower costs and less complexity. We’ve accomplished this by deploying a single-node Kubernetes cluster on a VM provider that is much cheaper, now running at 2i2c.mybinder.org. This both relieves Binder’s short-term capacity shortage and may provide an easier pathway for others to support the project in the future.

Below, we’ll describe what has changed to enable this, what we’re deploying, and what the impact should be.

Cloud infrastructure has become cheaper and more commodified #

A key theory of mybinder.org (and 2i2c) is that commercial cloud infrastructure will be commidified over time – what begins as cutting-edge functionality will become commonplace and offered across all cloud providers. As a result, costs will go down over time. Abstractions like Kubernetes will allow you to easily migrate workflows and infrastructure between cloud providers. As a result, you’ll be able to easily follow those costs where there are better options. That’s essentially what is happening here.

There are two key changes that make it much easier to deploy a BinderHub instance at a fraction of the cost:

First, Kubernetes has matured and become easier to deploy. When mybinder.org started, it was using the cutting-edge of Kubernetes functionality. This meant that we needed to use cloud providers that provided a managed Kubernetes service to deal with this complexity. A managed Kubernetes offering tends to be expensive, offered by only a few cloud providers, and thus raises costs across-the-board for the provider that offers it.

However, this was almost a decade ago, and Kubernetes has become both more functional and more stable. There are now many more ways of running Kubernetes, especially for simpler workflows that don’t require autoscaling. In the last several months, we’ve been experimenting with single-node Kubernetes workflows via K3s². K3s is a lightweight Kubernetes distribution that is much easier to deploy and manage. It’s designed for things like edge computing and low-resource environments, and it can be deployed with a single script!

By running a Kubernetes cluster on a single node, we don’t need a “managed Kubernetes service”, which means we can choose from a much larger pool of infrastructure / cloud providers. If all we need is a running VM, this is something the tech industry has been doing for decades.

Second, Managed Object Storage services have more open source options, and are more commodified and cheaper. In addition to Kubernetes, the other thing that BinderHub needs is a way to store and retrieve images for the environments that it builds. This also used to be a fairly complex problem, and thus required managed solutions from cloud providers that charged a premium for their service. However, a number of open source object storage solutions have emerged and made it much easier for providers to support this workflow.³. Because these are open source, infrastructure providers can provide managed object storage at a fraction of the cost.

Because of these two things, we’ve learned that we can run a BinderHub instance on a single VM from a much larger pool of infrastructure providers. This means we should be able to run BinderHub instances at a fraction of the cost.⁴

Deploying BinderHub on a single-node VM is cheaper and simpler #

Last week, we deployed 2i2c.mybinder.org, a single-node Kubernetes instance on Hetzner cloud using K3s. This will run on a single node VM, with a Kubernetes instance that is entirely managed by us, and with managed object storage from Hetzner. Compared to other cloud providers, it is around 5x cheaper per month.

Comparison of rough monthly costs across different cloud providers for similar VM instances. These are rough estimates based on cloud provider pricing pages for an on-demand VM with around 190GB RAM. Pricing pages: Hetzner Cloud ~$300, Microsoft Azure ~$1,300, Google Cloud Platform ~$1,500, Amazon Web Services ~$1,600.

Running a single-node Kubernetes instance will be a cheap and effective way to handle a lot of mybinder.org’s capacity needs. Because it’s a single node cluster, there is no auto-scaling (one reason it is so cheap), which reduces a lot of the complexity we’ll have to manage. These are acceptable tradeoffs for a service like mybinder.org, which runs entirely ephemeral sessions with very limited resources and no promises about uptime, persistence, etc.

You might be wondering: “I thought Kubernetes was supposed to save money.” Normally, running Kubernetes for scalable workflows does save costs because you can scale infrastructure to match your capacity needs. Without scaling, you’d need to provide a VM that can always handle your maximum capacity needs (and pay for the costs the entire time). With Kubernetes, you can request and remove nodes to grow your capacity as-needed (and save money doing so). It looks something like this:

The cost difference between a single large VM vs scalable nodes. Given variable usage over time, kubernetes allows you to scale your cost up and down with need, which is more efficient than paying for a single VM that can withstand your maximum capacity.

However, there is a built-in cost you pay when you use a service that provides managed Kubernetes. Managed Kubernetes services are complex and expensive, and this is reflected across-the-board in the provider’s costs. What if we could achieve the same outcome with a much simpler cloud offering like a single VM?

We did a bit of research and discovered that the Kubernetes and object storage landscape has indeed evolved significantly since the early days of mybinder.org. For example, Hetzner is a cloud provider that has been around for a long time. It has single-node VMs that are about 4x cheaper than their counterparts in Google Cloud or AWS, and provides managed object storage that uses MinIO in a cost-effective way. Using K3s, we can run a lightweight, single-node Kubernetes runtime on this node, and deploy a BinderHub with the same infrastructure as any other BinderHub federation member.

By our estimate, we could fit around 400 simultaneous sessions on mybinder.org (because each session uses very few cloud resources). This is already the majority of mybinder.org’s capacity needs, and at a much lower cost than using a scalable Kubernetes cluster. The cost picture looks something like this:

If your single VM is much cheaper, it might still be the cheapest option. In the case of a Hetzner VM, it has roughly the same capacity as another cloud provider’s VM, but at 1/4 of the cost.

2i2c.mybinder.org now serves 70% of the mybinder.org federation #

About a week ago, we launched 2i2c.mybinder.org running via the methodology we described above. We intended to run this as a longer experiment, but believe that it has already proven useful enough to consider “ready for production”. We recently increased 2i2c.mybinder.org’s load to 70% and will continue to monitor its performance over time. Here’s a plot of where each mybinder.org session has been run over the past ten days - you can see the moment where we turn on 2i2c.mybinder.org to the left:

Sessions launched on mybinder.org’s federation over the past ten days. The yellow area represents sessions run on 2i2c.mybinder.org. They now make up the majority of launches on mybinder.org. Prior to this, gesis.mybinder.org was the only remaining federation member.

For now, 2i2c is sponsoring a max of €350 a month (with some currency conversion noise) to run this service. We’ll provide in-kind labor to run this node, and treat it as an organizational investment in supporting open science, as well as learning new Kubernetes and cloud infrastructure workflows. We’re going to use funds recovered from communities in our community hub network, along with in-kind labor to build out this experiment.

In six months, we’ll evaluate how much effort it was to run this node for mybinder.org, whether it meaningfully helped with mybinder.org’s capacity, and whether it was sustainable for us from a time and labor perspective.

Others can join the mybinder.org federation using this approach as well #

We think that developing this single-node BinderHub workflow will make it much easier for others to join the mybinder.org federation, because it lowers the infrastructure and skills complexity needed to join. Here is a brief guide we’ve written for deploying a BinderHub with K3s. We are helping a few interested organizations deploy their own BinderHubs in this way in order to validate the idea, and are hopeful that this makes it much easier to grow mybinder.org’s capacity via new federation members.⁵

We’re excited to experiment with new ways to support mybinder.org. We think this is an excellent example of how open standards and technology lead to cloud workflows with lower costs and more flexibility. We also think it’s a good example of how it is valuable to have organizations aligned with open science (like 2i2c!) acting in this space. If you have any questions or comments, please join this Jupyter Zulip thread

Anybody want to fund this? #

If you’re interested in making open science infrastructure like Binder more scalable and sustainable, we’d love to find more resources to both sustain this node and cover more development time to run this experiment. Feel free to reach out here.

If you have access to VMs and object storage, and are interested in running a mybinder.org federation member using the methods described here, check out our brief guide for deploying a BinderHub with K3s.

If you’re generally interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the Support Binder page for how you can help.

If you’re interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the

Support Binder page for how you can help.

Acknowledgements #

Thanks to the JupyterHub community for helping us set up this new node.
Thanks to our member communities whose fees currently support this work.

Many thanks to GESIS and OVH for their continued support of mybinder.org, your contributions to keeping this service running are critical! ↩︎
thanks to Carl Boettiger for collaborating on this with us! ↩︎
One example is MinIO, which is used by Hetzner to provide managed object storage for their single-node VMs. ↩︎
For example, Hetzner provides a single-VM option with managed object storage that is roughly 25% of the cost of other cloud providers that also offer autoscaling Kubernetes services. There are many other infrastructure providers who could be used in this way. ↩︎
We’re also experimenting with a few other ways to reduce the complexity and costs of running a BinderHub even further, but will have more on that later as we learn more :-). ↩︎

Designing for an ecosystem: a case study in cross-project open source contribution

Tue, 21 Jan 2025 00:00:00 +0000

A key challenge in the open source space is that projects are often independent and autonomous, with relatively few formal ways to collaborate and coordinate efforts. While this usually isn’t a big deal, it means that there is a missed opportunity to grow the impact of an ecosystem because it requires coordinated development among multiple stakeholders within it.

This is one of the reasons we created 2i2c’s open community hub platform. By deploying a single platform that utilizes entirely open infrastructure that we contribute back to, we have visibility over a variety of projects along with the need to combine them together for a specific end-user outcome. One-such development scenario recently came up involving Jupyter Book 2 and JupyterHub.

Allowing readers to “bring their own Binders” #

We’ve recently been working to integrate Jupyter Book 2 workflows with our community hubs for a more seamless experience (for example, having book pages link back to interactive cloud sessions that allow users to interact with the content). We imagine a network of Jupyter Books that all build upon the same core infrastructures (JupyterHub, Binder, etc) for cloud-based computing. Our hope is to allow a user to bring their own Binder with them so that they can interact with another book’s content with their own cloud infrastructure. For example:

A student with access to binder.myuniversity.edu could read a Jupyter Book created by a professor at otheruniversity.edu.
The Jupyter Book is defined with a Binder specification that has a recipe for re-building the environment needed to run te book’s content.
From the professor’s book, the student can choose to launch an interactive Binder sessions on their university’s Binder, allowing them to interact with the book’s content on their own infrastructure.

We want a workflow like this to be as seamless and un-complicated as possible. We also want it to follow the same fundamental workflow as the nbgitpuller-based launch buttons. Along the way, we realized that we needed to coordinate development across Jupyter Book 2, JupyterHub, and BinderHub.

The three projects (Jupyter Book, BinderHub, and JupyterHub) that needed to work together to enable ‘bring your own binderhub’ workflows.

Getting Jupyter Book to discover Jupyter Hub #

As we began developing this workflow, we realized that there was a blocker in the JupyterHub and BinderHub ecosystem that needed to be fixed. We needed a way to ask a JupyterHub whether it had an unauthenticated end-point for service discovery. Basically, a way to ask a hub “what kind of hub are you, and how can we launch an interactive session on you?” Doing this is simple-enough - JupyterHub already has a way of reporting its version and application type, which allows us to infer how to launch interactive sessions. But, we hit a snag in an HTML context.

By default, JupyterHub disallows certain kinds of Cross-Origin Resource Sharing (CORS) requests, in order to restrict other applications from abusing a JupyterHub’s API. If you hit parts of a JupyterHub API from the command line, things work fine. But if you do the same thing via JavaScript from a website, the request is disallowed. This was a problem if we want Jupyter Book (a web application) to be able to make requests of JupyterHub’s API.

So, we realized that we needed to make an upstream contribution in JupyterHub in order to enable an interaction between JupyterHub and Jupyter Book. In this case, it was a relatively simple fix: allowing CORS requests for the specific API endpoint we needed (which is a very lightweight endpoint that is not vulnerable to security risks, and is broadly useful to make accessible)¹. That resulted in two PRs:

jupyterhub/jupyterhub#4966 allows CORS requests for the API that was needed for service discovery in JupyterHub.
jupyterhub/binderhub#1906 enables this workflow on a BinderHub so that its services can be discovered.
jupyter-book/myst-theme#503 adds new launch button functionality to Jupyter Book 2 that allows readers to bring their own Binder / JupyterHub links for launching. (this is what necessitated the above two PRs)

As a result of this upstream contribution loop, JupyterHub can now accept API requests at its “service discovery” endpoint, which means that Jupyter Book (and any other web application) can more easily learn about a hub’s capabilities and version.

We wanted to share this short vignette because it’s a good reflection of the kind of value that 2i2c tries to provide, given its role in helping to build and enhance networks of infrastructure, domain communities, and open source communities. In this case, we enabled a cross-project workflow that required knowledge of each project, and a vision for how they could be used together in a way that exceeded the sum of their parts.

We think there’s a lot more potential in these kinds of workflows, and are eager to continue our work to identify and enhance community-centric infrastructure for interactive computing.

Acknowledgements #

Thanks to the JupyterHub and Jupyter Book communities for collaboration and review on this work.

This actually required an interesting bit of team discussion that was much easier with a few 2i2c staff on the JupyterHub team. The original request from Angus was interpreted as opening up the entire hub API to external requests (which is a bad idea!) but we were able to quickly discuss this with the JupyterHub team to clarify that this was only about a very specific API endpoint. This is the kind of communication loop that often goes haywire when you have people contributing to a project without historical relationships to the project’s maintainers. ↩︎

NASA VEDA & 2i2c Update for Q4 2024 (Oct-Dec 2024)

Tue, 07 Jan 2025 15:18:37 -0800

A non-exhaustive list of things 2i2c and Development Seed did with the NASA VEDA project last quarter!

Automated backups and alerting with `jupyterhub-home-nfs` #

Tracking Issue

jupyterhub-home-nfs is a young project to provide flexible per-user home directory limits on JupyterHub - an important feature for controlling cloud costs. Tarashish Mishra and Sarah Gibson have been leading this project for the last few months. Since we are moving away from AWS Managed EFS here, we had to do some work to recreate some of the benefits EFS gives us out of the box. During this quarter, we:

Set up automated backups so we can recover files in cases of disaster
Set up automated alerting (via prometheus and pagerduty) to know if our backing EBS device is getting full and we need to perform a manual intervention
Deployed this to a few other communities ( CryoCloud and NMFS Openscapes) to broaden adoption.

We will continue doing work on jupyterhub-home-nfs in the upcoming quarter! If this is functionality you are interested in deploying, please reach out to us to collaborate!

Enable users to dynamically build environments with `jupyterhub-fancy-profiles` #

Tracking Issue

We covered this more extensively in another blog post, so go read that!

This work in particular is a good demonstrator of 2i2c’s value - it started off with a grant from GESIS, and now with support from NASA IMPACT we are able to bring it to a lot of communities, not just the ones that funded it.

Ongoing work here will focus on improving the UX as well as better documentation so users can actually use it!

“Open in QGIS” from VEDA UI #

Tracking Issue

We had worked in the past with many communities in enabling QGIS on the Cloud, and this quarter we got closer to enabling a contextual ‘Open in QGIS’ button in the VEDA Dashboard! Here is a quick demo:

(This shows the workflow when user is already logged into the JupyterHub and had started the server)

You can play with this in this preview, although you need to have access to the NASA VEDA hub to fully try it out at this point.

Tarashish from Development Seed is again responsible for most of the work here, available in jupyter-remote-qgis-proxy. You can use it to create ‘magic links’ that will open QGIS in a desktop environment in your browser, and add a specific layer to it! Our hope is that this allows primarily GIS folks to better use tools they already are familiar with in cloud based contexts.

Other updates #

We participated heavily in an evaluation process for the authentication and authorization solution to be used across NASA VEDA! Tracking Issue
We are very close to rolling out JupyterHub 5.0 and associated changes across all our hubs, which will enable us to eventually offer per-group shared directories! Tracking Issue

Acknowledgements #

Thanks to the NASA VEDA project for thir ongoing support for this work.
Thanks to DevSeet for their collaboration and leadership on this project.

Announcing the Jupyter Book 2 alpha

Mon, 18 Nov 2024 00:00:00 +0000

Cross-posted from the Jupyter Book blog. Note that some MyST functionality is not supported on the 2i2c website – please see the original post for previews.

Over the last ten months, the Jupyter Book team have been hard at work; Jupyter Book has become a Jupyter subproject, and the team¹ announced a plan to release Jupyter Book 2. This post announces the alpha release of Jupyter Book 2.0, which has been re-written from the ground up to use the new MyST-MD engine.

Over the next few months, we will work in preparation for the full release of Jupyter Book 2. Stay tuned for more! The initial documentation for the alpha release can be found at:

next.jupyterbook.org/

Install the Jupyter Book 2 Alpha #

The Jupyter Book 2 alpha is available from PyPI.org. You can install it with pip, using

pip install -U jupyter-book==2.0.0a0

If you use pipx, it’s recommened to run Jupyter Book 2 using

pipx run jupyter-book==2.0.0a0

Jupyter Book 2 needs Node.js installed on your computer. If this is not the case, running jupyter book will prompt you to install it using the nodeenv package that ships with Jupyter Book 2:

❗ Node.js (node) is required to run Jupyter Book, but could not be found`.
❔ Install Node.js in '...'? (y/N):

Press y and Enter to proceed.

The Jupyter Book 2 project is a complete re-write of Jupyter Book. We expect there to be bugs and breakages! Please use our support channels to keep us up to date with your findings!

Discord

GitHub Issues

New Features in `2.0 alpha` #

Rich Hover Previews #

The new MyST book and article themes provide useful hover previews for links to other MyST content, Wikipedia, GitHub issues, and many more.

Content from other websites built with the MyST engine can be embedded in your own sites and PDFs:

Cross-referenced content can easily be embedded and re-captioned into other pages and projects, such as this figure to mystmd.org/guide/embed#mylabel.

Simple Instant Search #

A new client-side search uses a simple, modern algorithm for fast local search that finds the results that you care about.

Client-side search uses simple, modern, Algolia-inspired search algorithm to provide useful search results. We will be iterating on this in the near future for even richer search results!

High Quality PDFs #

PDF documents can now be built with Typst, a high-quality typesetting engine that produces readable error messages and beautiful documents. This feature was the basis for the 2024 SciPy proceedings, which is now built on MyST Markdown and will be accepting Jupyter Notebooks in 2025.

Example of the LaPreprint Typst template for rendering PDFs from Jupyter Book (via the MyST Engine).

Coming Soon in `2.0 beta` #

Custom Styles & Scrips #

Jupyter Book 2 will make it easy to tweak your website styles, and add new website behaviors.

Generate Markdown from Code Cells #

The MyST engine is on-track to support the inclusion of references and other markup features generated by code cells.

Control Cell Visibility with Tags #

In the beta release, Jupyter Book 2 will once-again be able to show and hide content according to cell tags.

Jupyter Book 2 vs MyST-MD #

At this early stage, the new Jupyter Book application jupyter book behaves identically to the mystmd engine that it is built upon; as outlined in our Jupyter Book 2 plan, we intend for Jupyter Book to be an “opinionated distribution” of mystmd that shares the same configuration format and CLI. This contrasts with Jupyter Book 1, which was built on top of the Sphinx documentation engine, but offered its own CLI and configuration files. In future, the jupyter book and mystmd CLIs may diverge from one another, but we expect that this will be handled in a graceful manner: mystmd commands should always be compatible with the jupyter book application.

Acknowledgements #

Thanks to Project Pythia for funding some of our ongoing work on Jupyter Book.
Thanks to the Jupyter Book community for their collaboration.

Jupyter Book project has historically been a technical project of the Executable Books organisation. In 2024, the establishment of a Jupyter subproject means that the Jupyter Book project now has its own identity outside of Executable Books. ↩︎

Jupyter Book 2.0 will use MyST

Tue, 21 May 2024 00:00:00 +0000

See the Executable Books blog for a post on the future directions of the Jupyter Book project, which will be built on top of the MyST Markdown engine.

Acknowledgements #

This post relates to our ongoing collaboration with the Jupyter Book project.
Thanks to Project Pythia for funding part of our work on Jupyter Book.

2i2c supports Jupyter Docker Stacks ARM builds

Fri, 01 Dec 2023 00:00:00 +0000

The Jupyter Docker Stacks project provides a collection of ready-to-use Docker images for Jupyter environments. These images are used by many in the Jupyter community, including 2i2c which uses them as base images for our JupyterHub deployments.

The project recently began publishing ARM-compatible images alongside the standard x86 images, making it easier for users with ARM-based systems (like M1 Macs) to use these environments. However, building and hosting these ARM images comes with additional cloud computing costs that were being personally covered by @mathbunnyru, one of the project’s maintainers.

A part of 2i2c’s mission is supporting upstream communities that we rely on, especially where the upstream project has limited resources. For this reason, we’ve decided to support Jupyter Docker Stack’s ARM building costs, with a total budget of $2000 (approximately $150 per month). As a regular user and beneficiary of the Jupyter Docker Stacks, we believe it’s important to contribute to the maintenance and sustainability of this crucial piece of infrastructure that benefits the entire Jupyter community.

We hope this support helps the Docker Stacks project remain healthy, and continue providing high-quality, multi-architecture images that work across different computing platforms. We’ll revisit this decision as the landscape of technology providers changes and other options arise.

Acknowledgments #

Thanks to Project Jupyter (particularly the jupyter-stacks team) for this project.

On the Jupyter Blog: From intern to mentor.

Fri, 30 Jun 2023 00:00:00 +0000

6——1———————– #

GESIS - 2i2c collaborate to build a persistent BinderHub experience

Mon, 28 Nov 2022 00:00:00 +0000

Introduction #

Mybinder.org enables researchers across the world to replicate computational environments in the cloud. It allows researchers to turn static code into interactive literate coding environments with a click of a button within seconds. The mybinder.org service is powered by BinderHub, an open-source tool developed by the Jupyter Project that many organizations have deployed for their own communities. It does this by _dynamically building _the software environment needed to reproduce a computation (using a tool called repo2docker), and making this environment available to users.

BinderHub was developed for use-cases that are temporary and fully open by design. BinderHub sessions are destroyed after a fixed amount of time and there is no persistent storage or authentication. However, many research institutions also need more “standard” service features like authentication and persistent storage.

Over the past several years, the GESIS Notebooks team made the first steps towards bridging this gap through their Persistent BinderHub implementation. This was a modified and authenticated BinderHub that included persistent storage across sessions. The Persistent BinderHub service was very successful at GESIS and with its partner communities, and the team wishes to build this functionality into the JupyterHub community’s core technology so that these features can be enjoyed for more use-cases and by many communities.

To enable this vision, we have partnered with GESIS in cooperation with NFDI4DS (GAN: 460234259), CESSDA, and members of the JMTE project. This collaboration has three primary goals:

Generalize the Persistent BinderHub functionality/experience to run on cloud-agnostic infrastructure, so that other stakeholders in NFDI, CESSDA, and the broader scientific community may benefit from this functionality and experience.
Upstream this functionality by making contributions into Jupyter community projects, so that it will be maintained and improved by a community moving forward, thus improving its reliability and sustainability.
Improve the implementation and user experience around Persistent BinderHub, in order to make it more reliable, scalable, productive, and enjoyable to use.

We began this collaboration several months ago, and have focused our efforts on exploring potential implementation pathways for this functionality. We believe that we now have a path forward for this functionality, and this blog post is a brief report of our efforts and future plans as we undertake this effort. See this GitHub Projects Board for issues that implement this effort.

Exploration 1: Adding persistent storage directly into BinderHub #

Our initial intention was to incorporate persistent storage and authentication from the GESIS Persistent BinderHub into the BinderHub codebase. We began by holding a series of meetings to discuss technical requirements from our experience in the JupyterHub/BinderHub ecosystem, and also conducted an audit of the Persistent BinderHub codebase. The Persistent BinderHub implementation is a modified Helm Chart that configures a JupyterHub to expose its authentication and persistent storage functionality, overriding the BinderHub default behavior. We were concerned that building this functionality natively into BinderHub would be challenging given that the BinderHub codebase was designed for ephemeral user sessions.

So, we decided to take another approach:

Exploration 2: Add dynamic image building to JupyterHub #

We realized that there is a way to make this functionality more broadly useful and more maintainable, while still achieving the end-user experience that the GESIS team needed. Instead of modifying BinderHub to incorporate JupyterHub’s storage and authentication features, we would give JupyterHub the ability do dynamically generate user environments using repo2docker.

This would give JupyterHub users more flexibility over the environments served by their hub, and expose Binder-style workflows to the “typical” JupyterHub workflow. BinderHub could then be simplified to re-use JupyterHub’s image building functionality as a part of its own service. We also identified a prototype of this functionality in the tljh-repo2docker project that QuantStack had built for the PlasmaBio project. This implementation was seen as successful, and something others in the community had wanted to generalize for some time.

Our implementation plan #

Two phases of implementation #

With this alternative implementation route in place, we identified two major steps to accomplish this project:

Build a back-end for dynamic environment building. JupyterHub needs to understand how to call repo2docker’s image generation from a Docker-based environment. It needs to expose this ability via APIs that others can build interfaces on top of.
**Build a front-end that is user-friendly and accessible. **Once the back-end is functional, we must build a front-end experience that feels familiar to BinderHub users and is easy and intuitive to use.

Here are a few tasks that we’re carrying out next to make progress on the above two items.

Build a working prototype for image generation via a JupyterHub Service (see below for current status)
Research the tljh-repo2docker code base to understand how we could build upon its UX and functionality.
Understand the typical process that GESIS and NFDI users follow in their BinderHub workflows, to ensure that it can be replicated via this new implementation.
Perform UI/UX research validation to inform the implementation from a user’s perspective.

As a follow-up, we’ll likely re-work the BinderHub codebase to utilize JupyterHub’s new repo2docker service, rather than defining its own custom repo2docker functionality.

Back-end implementation as a JupyterHub Service #

We’re planning to use JupyterHub Services to add the back-end functionality for dynamic image building to a JupyterHub. Services are a way to expose functionality via a JupyterHub REST API, and would allow us to expose basic image generation on-the-fly with repo2docker. We aim for basic functionality to be as close as possible to repo2docker’s default behavior, but to make this functionality _composable and customizable _if a JupyterHub administrator wants to provide different out-of-the-box functionality.

While details may still change, we believe that the major architectural plan has now been settled. We have an experimental codebase with a basic implementation of the repo2docker service described above. As part of this effort, this team has also made a number of improvements to repo2docker’s codebase and project infrastructure. We hope that this effort will continue to push resources and attention to repo2docker by growing the number of users and stakeholders that rely on the project for their success.

Front-end implementation that uses this service #

Once the back-end setup exists, we can use it to begin prototyping user interactions that can trigger and use repo2docker’s image generation. Bringing dynamic environment image building into JupyterHub is a significant improvement in its functionality, and may introduce new kinds of workflows that we hadn’t initially imagined. Understanding, interpreting, and extending the original “Persistent BinderHub” workflow will require a more thorough understanding of user stories and needs in order to identify new workflows that feel natural not only in a JupyterHub scenario but also in a Binder-like scenario.

For example, here are a few major UI/UX questions we must answer:

When a user builds an image, should it also become available to other users as well?
How can a user store, find, and delete old images that they’ve built?
What about new versions of the same image?
Should we simply mimic the mybinder.org UX, or should this be improved as well?

We must answer these and several other questions next. With that vision in place, we’d like to expand our efforts in UI/UX and user research. This will help inform the technical implementation of this work as we expand on our prototype, and help us choose the right way to expose this functionality to users. We welcome collaboration around this work - if you know of an organization that is interested in collaboration, please reach out.

Collaborate and follow along #

Below you’ll find a rough project plan to give an idea for the major actions needed and a timeline for when we hope they’ll be completed. We’ll track further updates and progress on this project in this dedicated GitHub issue and this dedicated GitHub project board.

After a few months working on this project, we are even more excited about the potential for dynamically building environments in a JupyterHub. We believe that it adds a new class of workflows to JupyterHubs that were not possible before, and will be immediately useful to the hundreds of communities that deploy JupyterHub for their communities.

Upstream Impact | 2i2c

Supporting JupyterHub admins on workshop hubs with shared passwords

Acknowledgements #

Report from the Jupyter and AI community meetup

Jupyter AI and the Agent Client Protocol #

Notebook CLI #

Sidebar comments and Real-time collaboration #

Opportunities for 2i2c #

Acknowledgements #

Report from the Jupyter Security Working Group security tooling sprint

Acknowledgements #

Combining multiple repos into one site at jupyterbook.org

Acknowledgements #

Jenny Wong joins the JupyterHub team

Learn more #

Acknowledgements #

Introducing Jupyter Book 2 at FOSDEM 2026

Learn more #

Acknowledgements #

New Jupyter Book / MyST stack release (Jan 2026)

Where we contributed #

Acknowledgements #

Yuvi on scaling maintainer intuition to facilitate PR review with PR triage boards

Acknowledgements #

Creating a re-usable redirect generator for Jupyter Book 1 migrations

Learn more #

Refactoring Jupyter Book 2 documentation ahead of a major release

What we did #

Why we’re excited about it #

Learn more #

Acknowledgements #

Fixing the mybinder.org usage analytics archive

Learn more #

Acknowledgements #

Impact report from 2i2c's Binder federation instance

Usage over time #

Where we’ve made improvements #

Acknowledgements #

Combating tcp scanning on mybinder.org with the tcpflowkiller

Why this matters #

Learn more #

Acknowledgements #

TIL: GitHub Action secrets are only available from non-forked repositories

Our PR from a fork was using empty strings for repository secrets #

A quick fix: re-open the PR from the base repository #

How use secrets with forked repositories in a safe-ish way #

The pull_request_target workflow #

Using GitHub Environments for granular control #

We hope this saves you time! #

Learn more #

Updates from Chris' position on the Jupyter Executive Council and Foundation Board

Updates for the Jupyter community from Chris #

Blog posts about Chris’ experience on the executive council #

2i2c’s commitment to supporting open source communities #

Acknowledgements #

Sharing JupyterHub's vision for more flexible application deployment at the doepy talk series.

Learn more #

Acknowledgements #

Overhauling repo2docker's documentation

Acknowledgements #

2i2c's submissions to JupyterCon 2025

JupyterHub: A multi-user server for Jupyter notebooks #

Cloudy with a Chance of Savings: Per-User Usage and Cost Monitoring for JupyterHubs in the Cloud #

Controlling home directory costs (with user empathy) on the cloud with jupyterhub-home-nfs #

Rebuilding user trust in mybinder.org #

Can your JupyterHub handle your workload? Performance testing with jupyterhub-simulator #

Not just for notebooks: JupyterHub in 2025 #

Introducing Jupyter Book 2.0 #

You don’t need to contribute more to open source, you need to go to therapy #

What 2i2c has learned while trying to build sustainable relationships with Jupyter’s community. #

Open slides: Jupyter Book 2 and MyST at the UC Berkeley Data Science Education Summit

Learn more #

Acknowledgements #

Jupyter Book at the Scientific Python 2025 Developer Summit

Chris is joining Project Jupyter's Executive Council

Acknowledgements #

Simplifying and speeding up Binder builds with BuildKit

Acknowledgements #

2i2c joins the mybinder.org federation with a cheaper and faster way to deploy Binderhub

Cloud infrastructure has become cheaper and more commodified #

The `pull_request_target` workflow #

Rebuilding user trust in `mybinder.org` #

Can your JupyterHub handle your workload? Performance testing with `jupyterhub-simulator` #

Automated backups and alerting with `jupyterhub-home-nfs` #

Enable users to dynamically build environments with `jupyterhub-fancy-profiles` #

New Features in `2.0 alpha` #

Coming Soon in `2.0 beta` #