Tutorial | Sept. 20, 2020

Getting Started With GitHub Private Packages

How to deploy a private package to GitHub's shiny npm registry and use it within your CI.
Getting Started With GitHub Private Packages

Recently I've been in the need of publishing a private package. "How so?" you might ask. Another weird use-case of mine of course! Basically I had a set of helper functions that I needed to use both inside my project's core code but also with its related lambda functions. Those lambda would then be deployed on Netlify (because I still don't know how to use AWS Lambda directly, but that's fine), and if you're familiar with Netlify's functions you might know that once in production they aren't aware of anything outside their directory, so using something like this wouldn't work in production (but works with netlify dev locally which is misleading):

project/
├── functions/ # require("../helpers"); won't work in production
├── helpers/
└── src/

One of the cool things with Netlify's functions though is that they are aware of project's dependencies! So my idea was to actually turn this helpers folder into a proper package, install it as a project dependency, and then both my source code and functions would have access to its content:

project/
├── functions/ # 3. require("helpers"); now work in production (ノ◕ヮ◕)ノ*:・゚✧
├── helpers/
│   └── package.json # 1. helpers is made a proper package
├── src/
└── package.json # 2. helpers is then installed as a project dependency

Now that I had that figured out I needed to publish this package, and as a private one for two reasons:

  • Those helpers were really specific to that project, I don't see how I or anyone else would have used them in another project;
  • This project was closed source, so it didn't make sense to have this part only opened to public.

And here we go, that's how I started to look for a way to publish this package as a private one. At first, I checked npm registry, of course, but this is a feature available on their paid plans only and I didn't want to pay $7 per month just to host one package privately. So I looked at GitHub which I knew started to offer a package feature last year, discovered that it comes with 500 MB of free private packages for everyone, more than needed, and that's basically how I went the GitHub Packages way, and why you're now to read this blog post~

OK, I'm done telling everyone my life, let's see how to actually publish a private package on GitHub's registry...

Preparing a Package

Well, as obvious as it sounds you need a package to publish in order to... publish it. Just make sure you have one already or create a dummy one if you just want to test it yourself. You should have a package.json file with at least those two keys:

{
  "name": "helpers",
  "version": "1.0.0"
}

This is already enough if you want to publish your package to npm as long as its name is available (spoiler alert, helpers is taken), but we have to update few things in order to have it published on GitHub's registry instead of npm's.

First, we need to tell npm: "Oh, wait, this one is going to GitHub's registry." To do so there's the handy publishConfig key that allows us to configure such behavior directly inside our package.json, see by yourself:

{
  "name": "helpers",
  "version": "1.0.0",
  "publishConfig": {
    "registry": "https://npm.pkg.github.com" // This is GitHub's registry endpoint
  }
}

Then we need to specify the repository where this package's code is (or will be) living. GitHub requires it because it matches packages to repositories. Again here there's a repository key that we can configure inside our package.json file:

{
  "name": "helpers",
  "version": "1.0.0",
  "publishConfig": {
    "registry": "https://npm.pkg.github.com"
  },
  "repository": {
    "type": "git",
    "url": "ssh://git@github.com/lihbr/project.git" // A link to your repository
    "directory": "helpers" // If your package is not available at repository root
                           // add the path to it using this directory key~
  }
}

Voila~ "We should be good with our package.json now?" Well, one last thing:

GitHub Packages only supports scoped npm packages. (GitHub's documentation)

What does that mean? A "scoped package" is a package with a name matching this format: @owner/name. Let's break it down quickly:

  • @ is an identifier for scoped packages, they always start with this symbol;
  • owner is either your account name or the name of an organization you're part of;
  • name is the desired name of the package, separated from previous owner with a /.

Taking that into account let's update our package.json one last time:

{
  "name": "@lihbr/helpers", // `lihbr` being my account name here
  "version": "1.0.0",
  "publishConfig": {
    "registry": "https://npm.pkg.github.com"
  },
  "repository": {
    "type": "git",
    "url": "ssh://git@github.com/lihbr/project.git"
    "directory": "helpers"
  }
}

We're done preparing our package for being published on GitHub. Let's now see how to actually publish it.

Publishing our Package

OK, so everything is ready, let's run $ npm publish and...

npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="GitHub Package Registry"

Indeed, we forgot to authenticate to GitHub Packages! To do so need to get a personal access token. Here's how to get one:

  1. Head to GitHub and log in;
  2. Once logged in, click your profile picture at the top right of the screen, then "Settings";
  3. On the settings page pick "Developer settings" right at the bottom of the side navigation;
  4. Now pick "Personal access tokens" in the side navigation again.

You should now be on a page like that, if logged in you can reach it directly through this link:

GitHub personal access tokens settings page

Click "Generate new token". I like to have one per machine or per service I'm using but it's up to you to manage them the way you like. Fill the "Note" field with something that can actually help you in the future figuring out where this token is being used, and pick at least the following scopes for usage with GitHub Packages:

GitHub personal access tokens scopes for packages: repo, write:packages, read:packages, delete:packages

You can then click "Generate token" at the bottom of the screen and GitHub will prompt you once the generated token, so make sure to get it in your clipboard or you'll have to repeat the above process.

Now that we have a personal access token we can authenticate to GitHub's registry with the $ npm login command like this:

$ npm login --registry=https://npm.pkg.github.com
> Username: # Your GitHub username
> Password: # The personal access token we just generated, NOT your GitHub password!
> Email: # A public email address

After you logged in let's try to run again the first $ npm publish command. If everything went well, you should see your newly published package from its source repository homepage (bottom of the sidebar):

Packages list from a GitHub repository

Public or Private Package?

You might be wondering: "Hmm, I published my package, but is it publicly available?" With GitHub the answer is pretty simple: it depends on its related repository visibility. Public? Your package is public. Private? It's not. The only exception being if you publish your package when your repository is public and then switch it to private, in that case your package will remain public to prevent breaking other projects depending on it, fair enough.

OK, so, so far we learned how to create and configure a package for GitHub Packages and how to actually publish it there, that's great! But the point of a package is to actually being able to install it...

Installing Packages From GitHub Packages

Again, here just throwing $ npm install @lihbr/helpers (using your package name of course) in your console won't work out of the box. We have to tell npm to actually fetch our packages published on GitHub from GitHub's registry and not npm's.

To do so npm (and Yarn) makes use of .npmrc files (more on them on npm's documentation), there's one already at your home directory and if you peek into it you should see your GitHub personal access token that we used before, but anyway, here we'll create another one at a project root where we want to use our new shiny package with the following content:

@lihbr:registry=https://npm.pkg.github.com

See that @lihbr? That's the scope of our package we defined before, replace it with yours obviously. Basically this line is just telling npm "'Fetch every package with that scope on this registry instead of the default one", and if you have multiple packages with different scopes that you need to fetch from GitHub's registry (or other registries) you can just append other lines following the same pattern.

Once you have your .npmrc set up at a project root you should now be able to install your package locally, awesome! "But what about my CI now?" In case you were wondering if you need to version this .npmrc file, you definitely do because it'll allow you, your colleagues (yes, think about them also), and your CI to clone your repository and benefit from its customized npm configuration, therefore allowing them to install your packages published on GitHub Packages. With that everything should work out of the box for them, except... except if your package is private.

Handling Private Packages

Yes, that's why I used GitHub Packages for in first place, and so far installing private packages from GitHub locally has been working for us because we already logged in into its registry when we needed to publish our package. Let's see how to authenticate our CI to GitHub's registry to also allow it to install our private packages...

Here again, we'll make use of our .npmrc file at our project root, the one that is versioned. Let's add another line to it:

@lihbr:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}

This new line will tell npm to authenticate to the given registry, here GitHub one, with the provided token, but we don't append our token directly in this file as it's a versioned one. To work that around we make use of the ${GITHUB_TOKEN} syntax which npm will interpolate and replace with the value of our GITHUB_TOKEN environment variable here, and now I'm pretty sure you start to see where this is all going...

Head to your CI configuration, register another environment variable named GITHUB_TOKEN and give it the value of the personal access token we created before together (alternatively you can create another one just for your CI, with only read:packages as a scope). Now when you spin your CI it should be able to install your private packages also, great!

One last gotcha though, since we told npm to authenticate to a registry through an environment variable inside our .npmrc file at the project root, it now expects that GITHUB_TOKEN environment variable when developing locally too. Here's an example on how to set it on Windows, Mac, or Linux, just in case, but you're the boss regarding how you manage them on your OS of course:

# On Windows
$ setx GITHUB_TOKEN 9cecfe9470b9fa45fa08c866e0cf912c905e80d7
# On MacOS or Linux
$ echo "export GITHUB_TOKEN=9cecfe9470b9fa45fa08c866e0cf912c905e80d7" >> ~/.bash_profile

Summing Up

Well, that's all about it! We covered how to publish a public or private package to GitHub's registry, how to allow its installation on local and CI environments, as well as what are the involvement for our project configuration.

I hope that everything was clear enough, if not please reach out to me on Twitter! And in case you want to dig deeper into GitHub Packages here's a link to their official documentation~

Thanks for reading!

Like what you read?
がおがお