Devshell Files Maker

About

Helps static file/configuration creation with Nix and devshell.

There is a bunch of ways static file/configuration are hard, this will help you generate, validate and distribute JSON, YAML, TOML or TXT.

Generate

Your content will be defined in Nix Language, it means you can use variables, functions, imports, read files, etc.

The modular system helps layering configurations, hiding complexity and making it easier for OPS teams.

Validate

Your content modules could optionally be well defined and type checked in build proccess with this same tool.

Or you could use Nix as package manager and install any tool to validate your configuration (ie integrating it with existing JSON Schema).

Distribute

Nix integrates well with git and http, it could be also used to read JSON, YAML, TOML, zip and gz files.

In fact Nix isn't a configuration tool but a package manger, we are only using it as configuration tool because the language is simple and flexible.

You can recreate files of a repository directly to your local machine by running nix develop <flake-uri> --build, example:

# copy all my dogfood to your current folder nix develop github:cruel-intentions/devshell-files --build

With help of Nix and devshell you could install any development or deployment tool of its 80 000 packages.

Instructions

Installing Nix

curl -sSf -L https://install.determinate.systems/nix | sh -s -- install

Configuring new projects:

nix flake new -t github:cruel-intentions/devshell-files my-project cd my-project git init git add *.nix flake.lock

Configuring existing projects:

nix flake new -t github:cruel-intentions/devshell-files ./ git add *.nix git add flake.lock

Generating files:

nix develop --build

or entering in shell with all commands and alias

nix develop -c $SHELL # to list commands and alias # now run: menu

Examples

Creating JSON, TEXT, TOML or YAML files

# examples/hello.nix # # this is one nix file { files.json."/generated/hello.json".hello = "world"; files.toml."/generated/hello.toml".hello = "world"; files.yaml."/generated/hello.yaml".hello = "world"; files.hcl."/generated/hello.hcl".hello = "world"; files.text."/generated/hello.txt" = "world"; }

Your file can be complemented with another module

# examples/world.nix # almost same as previous example # but show some language feature let name = "hello"; # a variable in { files = { json."/generated/${name}.json".baz = ["foo" "bar" name]; toml."/generated/${name}.toml".baz = ["foo" "bar" name]; yaml = { "/generated/${name}.yaml" = { baz = [ "foo" "bar" name ]; }; }; }; }

Content generated by those examples are in generated

# ie ./generated/hello.yaml baz: - foo - bar - hello hello: world

Dogfooding

This project is configured by module project.nix

# ./project.nix { # import other modules imports = [ ./examples/hello.nix ./examples/world.nix ./examples/readme.nix ./examples/gitignore.nix ./examples/license.nix ./examples/interpolation.nix ./examples/docs.nix ./examples/book.nix ./examples/services.nix ./examples/nim.nix ./examples/nushell.nix ./examples/watch.nix ]; # My shell name devshell.name = "devshell-files"; # install development or deployment tools packages = [ "convco" # now we can use 'convco' command https://convco.github.io # but could be: # "awscli" # "azure-cli" # "cargo" # "conda" # "go" # "nim" # "nodejs" # "nodejs-18_x" # "nushell" # "pipenv" # "python39" # "ruby" # "rustc" # "terraform" # "yarn" # look at https://search.nixos.org for more packages ]; # create alias files.alias.feat = ''convco commit --feat $@''; files.alias.fix = ''convco commit --fix $@''; files.alias.docs = ''convco commit --docs $@''; files.alias.alou = ''  #!/usr/bin/env python  print("Alo!") # is hello in portuguese  ''; # now we can use feat, fix, docs and alou commands # create .envrc for direnv files.direnv.enable = true; # disabe file creation when entering in the shell # call devshell-files instead # files.on-call = true; }

This README.md is also a module defined as above

# There is a lot things we could use to write static file # Basic intro to nix language https://github.com/tazjin/nix-1p # Some nix functions https://teu5us.github.io/nix-lib.html {lib, ...}: { files.text."/README.md" = builtins.concatStringsSep "\n" [ "# Devshell Files Maker" (builtins.readFile ./readme/toc.md) (builtins.readFile ./readme/about.md) (builtins.readFile ./readme/installation.md) (builtins.import ./readme/examples.nix) ((builtins.import ./readme/modules.nix) lib) (builtins.readFile ./readme/todo.md) (builtins.readFile ./readme/issues.md) (builtins.readFile ./readme/seeAlso.md) ]; }

Our .gitignore is defined like this

# ./examples/gitignore.nix { # create my .gitignore copying ignore patterns from # github.com/github/gitignore files.gitignore.enable = true; files.gitignore.template."Global/Archives" = true; files.gitignore.template."Global/Backup" = true; files.gitignore.template."Global/Diff" = true; files.gitignore.pattern."**/.data" = true; files.gitignore.pattern."**/.direnv" = true; files.gitignore.pattern."**/.envrc" = true; files.gitignore.pattern."**/.gitignore" = true; files.gitignore.pattern."**/flake.lock" = true; }

And our LICENSE file is

# ./examples/license.nix { # LICENSE file creation # using templates from https://github.com/spdx/license-list-data files.license.enable = true; files.license.spdx.name = "MIT"; files.license.spdx.vars.year = "2023"; files.license.spdx.vars."copyright holders" = "Cruel Intentions"; }

Writing new modules

Nix lang

Jump this part if aready know Nix Lang, if don't there is a small concise content of Nix Lang.

If one page is too much to you, the basic is:

: defines a new function, arg: "Hello ${arg}"
that's why we use = instaed of :, { attr-key = "value"; }
; instead of , and they aren't optional
array aren't separated by , ie. [ "some" "value" ]

JSON as NIX

name	JSON	NIX
null	`null`	`null`
bool	`true`	`true`
int	`123`	`123`
float	`12.3`	`12.3`
string	`"string"`	`"string"`
array	`["some","array"]`	`["some" "array"]`
object	`{"some":"value"}`	`{ some = "value"; }`
multiline-string		`''... multiline string ... ''`
variables		`let my-var = 1; other-var = 2; in my-var + other-var`
function		`my-arg: "Hello ${my-arg}!"`
variable-function		`let my-function = my-arg: "Hello ${my-arg}!"; in ...`
calling-a-function		`... in my-function "World"`

Module

Modules can be defined in two formats:

As attrset, aka. object (JSON), dict (Python):

{ # <| imports = []; # | config = {}; # | module info options = {}; # | } # <|

All those attributes are optional

imports: array with paths to other modules
config: object with actual configurations
options: object with our config type definition

As function:

Functions has following arguments:

config with all evaluated configs values,
pkgs with all nixpkgs available.
lib library of useful functions.
And may receive others (we use ... to ignore them)

{ config, pkgs, lib, ... }: # <| function args { # <| imports = []; # | config = {}; # | module info options = {}; # | } # <|

Imports

Points to other modules files to be imported in this module

{ imports = [ ./gh-actions-options.nix ./gh-actions-impl.nix ]; }

Hint, split modules in two files:

One mostly with options, where your definition goes
Other with config, where your information goes

It has two advantages, let share options definitions across projects more easily.

And it hides complexity, hiding complexity is what abstraction is all about, we didn't share options definitions across projects to type less, but because we could reuse an abstraction that helps hiding complexity.

Config

Are values to our options

We can set value by ourself, or use lib functions to import json/toml/text files.

{ lib, ...}: { config.files.text."/HW.txt" = "Hello World!"; config.files.text."/EO.txt" = lib.concatStringsSep "" ["48" "65" "6c" "6c" "6f"]; config.files.text."/LR.txt" = (lib.importJSON ./hello.json).msg; # { "msg": "Hello World!" } config.files.text."/LL.txt" = (lib.importTOML ./hello.toml).msg; # msg = Hello World! config.files.text."/OD.txt" = lib.readFile ./hello.txt; # Hello World! }

If file has no options., config. can be ommited.

And this file produce the same result

{ lib, ...}: { files.text."/HW.txt" = "Hello World!"; files.text."/EO.txt" = lib.concatStringsSep "" ["48" "65" "6c" "6c" "6f"]; files.text."/LR.txt" = (lib.importJSON ./hello.json).msg; # { "msg": "Hello World!" } files.text."/LL.txt" = (lib.importTOML ./hello.toml).msg; # msg = Hello World! files.text."/OD.txt" = lib.readFile ./hello.txt; # Hello World! }

Options

Options are schema definition for configs values.

Example, to create a github action file, it could be done like this:

{ config.files.yaml."/.github/workflows/ci-cd.yaml" = { on = "push"; jobs.ci-cd.runs-on = "ubuntu-latest"; jobs.ci-cd.steps = [ { uses = "actions/checkout@v2.4.0"; } { run = "npm i"; } { run = "npm run build"; } { run = "npm run test"; } { run = "aws s3 sync ./build s3://some-s3-bucket"; } ]; }; }

This only works because this project has another module with:

{lib, ...}: { options.files = submodule { options.yaml.type = lib.types.attrsOf lib.types.anything; }; }

But if we always set ci-cd.yaml like that, no complexity has been hidden, and requires copy and past it in every project.

Since most CI/CD are just: 'Pre Build', 'Build', 'Test', 'Deploy'

What most projects really need is something like:

# any module file (maybe project.nix) { # our build steps config.gh-actions.setup = "npm i"; config.gh-actions.build = "npm run build"; config.gh-actions.test = "npm run test"; config.gh-actions.deploy = "aws s3 sync ./build s3://some-s3-bucket"; }

Adding this to project.nix, throws an error undefined config.gh-actions, and command fails.

It doesn't knows these options.

To make aware of it, we had to add options schema of that.

# gh-actions-options.nix { lib, ...}: { # a property 'gh-actions.setup' options.gh-actions.setup = lib.mkOption { default = "echo setup"; description = "Command to run before build"; example = "npm i"; type = lib.types.str; }; # a property 'gh-actions.build' options.gh-actions.build = lib.mkOption { default = "echo build"; description = "Command to run as build step"; example = "npm run build"; type = lib.types.str; }; # a property 'gh-actions.test' options.gh-actions.test = lib.mkOption { default = "echo test"; description = "Command to run as test step"; example = "npm test"; type = lib.types.str; }; # a property 'gh-actions.deploy' options.gh-actions.deploy = lib.mkOption { default = "echo deploy"; description = "Command to run as deploy step"; example = "aws s3 sync ./build s3://my-bucket"; type = lib.types.lines; }; }

Or using lib.types.fluent

# gh-actions-options.nix { lib, ...}: lib.types.fluent { options.gh-actions.options = { # defines a property 'gh-actions.setup' setup.default = "echo setup"; #default is string setup.mdDoc = "Command to run before build"; setup.example = "npm i"; # defines a property 'gh-actions.build' build.default = "echo build"; build.mdDoc = "Command to run as build step"; build.example = "npm run build"; # defines a property 'gh-actions.test' test.default = "echo test"; test.mdDoc = "Command to run as test step"; test.example = "npm test"; # defines a property 'gh-actions.deploy' deploy.default = "echo deploy"; deploy.mdDoc = "Command to run as deploy step"; deploy.example = "aws s3 sync ./build s3://my-bucket"; deploy.type = lib.types.lines; }; }

Now, previous config can be used, but it does nothing, it doesn't create yaml.

It knowns what options can be accepted as config, but not what to do with it.

The following code uses parameter config that has all evaluated config values.

# gh-actions.nix { config, lib, ... }: { imports = [ ./gh-actions-options.nix ]; # use other module that simplify file creation to create config file files.yaml."/.github/workflows/ci-cd.yaml".jobs.ci-cd.steps = [ { uses = "actions/checkout@v2.4.0"; } { run = config.gh-actions.setup; } #  { run = config.gh-actions.build; } # Read step scripts from { run = config.gh-actions.test; } # config.gh-actions { run = config.gh-actions.deploy"; } #   ];  files.yaml."/.github/workflows/ci-cd.yaml".on = "push";  files.yaml."/.github/workflows/ci-cd.yaml".jobs.ci-cd.runs-on = "ubuntu-latest"; }

Now it can be imported and set 'setup', 'build', 'test' and 'deploy' configs

# any other module file, maybe project.nix { imports = [ ./gh-actions.nix ]; gh-actions.setup = "echo 'paranaue'"; gh-actions.build = "echo 'paranaue parana'"; gh-actions.build = "echo 'paranaue'"; gh-actions.deploy = ''  echo "paranaue   parana"  ''; }

If something that is not a string is set, an error will raise, cheking it against the options schema.

There are other types that can be used (some of them):

lib.types.bool
lib.types.path
lib.types.package
lib.types.int
lib.types.ints.unsigned
lib.types.ints.positive
lib.types.ints.port
lib.types.ints.between
lib.types.str
lib.types.lines
lib.types.enum
lib.types.submodule
lib.types.nullOr (typed nullable)
lib.types.listOf (typed array)
lib.types.attrsOf (typed hash map)
lib.types.uniq (typed set)

And lib has some modules helpers functions like:

lib.mkIf : to only set a property if some informaiton is true
lib.optionals : to return an array or an empty array
lib.optionalString: to return an string or an empty string

Sharing our module

Now to not just copy and past it everywhere, we could create a git repository, ie. gh-actions

Then we could let nix manage it for us adding it to flake.nix file like

{ description = "Dev Environment"; inputs.dsf.url = "github:cruel-intentions/devshell-files"; inputs.gha.url = "github:cruel-intentions/gh-actions"; # for private repository use git url # inputs.gha.url = "git+ssh://git@github.com/cruel-intentions/gh-actions.git"; outputs = inputs: inputs.dsf.lib.mkShell [ "${inputs.gha}/gh-actions.nix" ./project.nix ]; }

Or manage version adding it directly to project.nix (or any other module file)

{ imports = let gh-actions = builtins.fetchGit { url = "git+ssh://git@github.com/cruel-intentions/gh-actions.git"; ref = "master"; rev = "46eead778911b5786d299ecf1a95c9ed4c130844"; }; in [ "${gh-actions}/gh-actions.nix" ]; }

Document our module

To document our modules is simple, we just need to use config.files.docs as follow

# examples/docs.nix {lib, pkgs, ...}: { files.docs."/gh-pages/src/modules/alias.md".modules = [ ../modules/alias.nix ../modules/alias-complete.nix ]; files.docs."/gh-pages/src/modules/cmds.md".modules = [ ../modules/cmds.nix ]; files.docs."/gh-pages/src/modules/files.md".modules = [ ../modules/files.nix ]; files.docs."/gh-pages/src/modules/git.md".modules = [ ../modules/git.nix ]; files.docs."/gh-pages/src/modules/on-call.md".modules = [ ../modules/startup.nix ]; files.docs."/gh-pages/src/modules/gitignore.md".modules = [ ../modules/gitignore.nix ]; files.docs."/gh-pages/src/modules/hcl.md".modules = [ ../modules/hcl.nix ]; files.docs."/gh-pages/src/modules/json.md".modules = [ ../modules/json.nix ]; files.docs."/gh-pages/src/modules/mdbook.md".modules = [ ../modules/mdbook.nix ]; files.docs."/gh-pages/src/modules/nim.md".modules = [ ../modules/nim.nix ]; files.docs."/gh-pages/src/modules/nushell.md".modules = [ ../modules/nushell.nix ]; files.docs."/gh-pages/src/modules/nush.md".modules = [ ../modules/nush.nix ../modules/nuon.nix ]; files.docs."/gh-pages/src/modules/rc.md".modules = [ ../modules/services/rc-devshell.nix ]; files.docs."/gh-pages/src/modules/services.md".modules = [ ../modules/services.nix ]; files.docs."/gh-pages/src/modules/spdx.md".modules = [ ../modules/spdx.nix ]; files.docs."/gh-pages/src/modules/text.md".modules = [ ../modules/text.nix ]; files.docs."/gh-pages/src/modules/toml.md".modules = [ ../modules/toml.nix ]; files.docs."/gh-pages/src/modules/watch.md".modules = [ ../modules/watch ]; files.docs."/gh-pages/src/modules/yaml.md".modules = [ ../modules/yaml.nix ]; }

We could also generate a mdbook with it

# examples/book.nix {lib, ...}: let project = "devshell-files"; author = "cruel-intentions"; org-url = "https://github.com/${author}"; edit-path = "${org-url}/${project}/edit/master/examples/{path}"; in { files.mdbook.authors = ["Cruel Intentions <${org-url}>"]; files.mdbook.enable = true; files.mdbook.gh-author = author; files.mdbook.gh-project = project; files.mdbook.language = "en"; files.mdbook.multilingual = false; files.mdbook.summary = builtins.readFile ./summary.md; files.mdbook.title = "Nix DevShell Files Maker"; files.mdbook.output.html.edit-url-template = edit-path; files.mdbook.output.html.fold.enable = true; files.mdbook.output.html.git-repository-url = "${org-url}/${project}/tree/master"; files.mdbook.output.html.no-section-label = true; files.mdbook.output.html.site-url = "/${project}/"; files.gitignore.pattern.gh-pages = true; files.text."/gh-pages/src/introduction.md" = builtins.readFile ./readme/about.md; files.text."/gh-pages/src/installation.md" = builtins.readFile ./readme/installation.md; files.text."/gh-pages/src/examples.md" = builtins.import ./readme/examples.nix; files.text."/gh-pages/src/modules.md" = "## Writing new modules"; files.text."/gh-pages/src/nix-lang.md" = builtins.readFile ./readme/modules/nix-lang.md; files.text."/gh-pages/src/json-nix.md" = builtins.import ./readme/modules/json-vs-nix.nix lib; files.text."/gh-pages/src/module-spec.md" = builtins.readFile ./readme/modules/modules.md; files.text."/gh-pages/src/share.md" = builtins.readFile ./readme/modules/share.md; files.text."/gh-pages/src/document.md" = builtins.import ./readme/modules/document.nix; files.text."/gh-pages/src/builtins.md" = builtins.readFile ./readme/modules/builtins.md; files.text."/gh-pages/src/todo.md" = builtins.readFile ./readme/todo.md; files.text."/gh-pages/src/issues.md" = builtins.readFile ./readme/issues.md; files.text."/gh-pages/src/seeAlso.md" = builtins.readFile ./readme/seeAlso.md; files.alias.publish-as-gh-pages-from-local = ''  # same as publish-as-gh-pages but works local  cd $PRJ_ROOT  ORIGIN=`git remote get-url origin`  cd gh-pages  mdbook build  cd book  git init .  git add .  git checkout -b gh-pages  git commit -m "docs(gh-pages): update gh-pages" .  git remote add origin $ORIGIN  git push -u origin gh-pages --force  ''; }

And publish this mdbook to github pages with book-as-gh-pages alias.

Builtin Modules

Builtin Modules are modules defined with this same package.

They are already included when we use this package.

files.alias, create bash script alias
files.cmds, install packages from nix repository
files.docs, convert our modules file into markdown using nmd
files.git, configure git with file creation
files.on-call, connfigure file to created only when devshell-files command is called, not on shell start
files.gitignore, copy .gitignore from templates
files.hcl, create HCL files with nix syntax
files.json, create JSON files with nix syntax
files.mdbook, convert your markdown files to HTML using mdbook
files.nim, similar to files.alias, but compiles Nim code
files.nus, similar to files.alias, but runs in Nushell
files.nush, similar to files.nus, but for subcommands
files.services, process supervisor for development services using s6
files.rc , WIP, process supervisor for development services using s6-rc
files.spdx, copy LICENSE from templates
files.text, create free text files with nix syntax
files.toml, create TOML files with nix syntax
files.watch, create an alias and service to run command when file changes using inotify-tools
files.yaml, create YAML files with nix syntax

Our documentation is generated by files.text, files.docs and files.mdbook

TODO

Add modules for especific cases:
- ini files
- most common ci/cd configuration
Verify if devshell could add it as default

Issues

This project uses git as version control, if your are using other version control system it may not work.

Don't look at

*nix general definition of Unix/Linux
Nixos.com NSFW
Nixos.com.br furniture

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Devshell Files Maker

About

Generate

Validate

Distribute

Instructions

Generating files:

Examples

Dogfooding

Writing new modules

Nix lang

JSON as NIX

Module

As attrset, aka. object (JSON), dict (Python):

As function:

Imports

Config

Options

Sharing our module

Document our module

Builtin Modules

TODO

Issues

See also

Don't look at

About

Uh oh!

Releases

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 317 Commits
examples		examples
generated		generated
modules		modules
templates/default		templates/default
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
dockerfile		dockerfile
flake.lock		flake.lock
flake.nix		flake.nix
project.nix		project.nix

Folders and files

Latest commit

History

Repository files navigation

Devshell Files Maker

About

Generate

Validate

Distribute

Instructions

Generating files:

Examples

Dogfooding

Writing new modules

Nix lang

JSON as NIX

Module

As attrset, aka. object (JSON), dict (Python):

As function:

Imports

Config

Options

Sharing our module

Document our module

Builtin Modules

TODO

Issues

See also

Don't look at

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Contributors

Uh oh!

Languages