Gemini proposed this:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

What I ended up doing:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I notice LLMs tend to err on the side of complexity. I don’t know if this is due to their training to spit out as many tokens as possible, but using regular expressions when a simple string replacement may suffice is a good sign that they may be adding more complexity than necessary.

Gemini proposed this:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

What I ended up doing:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I notice LLMs tend to err on the side of complexity. I don’t know if this is due to their training to spit out as many tokens as possible, but using regular expressions when a simple string replacement may suffice is a good sign that they may be adding more complexity than necessary.

Gemini proposed this:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

What I ended up doing:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I notice LLMs tend to err on the side of complexity. I don’t know if this is due to their training to spit out as many tokens as possible, but using regular expressions when a simple string replacement may suffice is a good sign that they may be adding more complexity than necessary.

Gemini proposed this:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

What I ended up doing:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I notice LLMs tend to err on the side of complexity. I don’t know if this is due to their training to spit out as many tokens as possible, but using regular expressions when a simple string replacement may suffice is a good sign that they may be adding more complexity than necessary.

It proposed the following:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

However, I ended up using this simplified version:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I’ve noticed that LLMs often lean toward overly complex solutions. I’m not sure if this is due to being trained to generate as many tokens as possible, but using regular expressions where a simple string replacement would suffice is often a sign of unnecessary complexity.

Gemini proposed this:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -r '\{\+|\+\}' '' | str join " " | str words | length

What I ended up doing:

git diff --word-diff=plain -- "*.md" | rg -o '\{\+(.*?)\+\}' | str replace -a "{+" "" | str replace -a "+}" "" | str replace -a ' ' "\n" | lines | length

I notice LLMs tend to err on the side of complexity. I don’t know if this is due to their training to spit as many tokens as possible, but using regular expressions when a simple single replace may suffice is a good sign that they may be adding more complexity than necessary.

set -g default-shell nu

in .tmux.conf to specify Nushell as the shell, it doesn’t load $env.PATH. It is necessary to specify the shell with:

set -g default-command nu -l

(using the full path to nu) to ensure the shell reads your configuration file.

virtualenv .venv
overlay use .venv/bin/activate.nu

🐇 https://www.getzola.org/themes/pico/ is an option, but I don’t like its header.

🦊 Minimal Dark from the same author looks better: https://kuznetsov17.github.io/minimal-dark/notes/note1/

🐇 https://www.getzola.org/themes/no-style-please/ is also an option.

🦊 https://halve-z.netlify.app/posts/information/ looks interesting, but there is too much screen estate for the left bar.

🐢 Let’s start by running the site locally first.

🐇 There are errors in the configuration. That’s weird, but let’s fix these.

🐢 Fixed errors. These are probably related to a newer version. Can we take a look at netlify.toml to see if it downloads the same version?

🐇 There are breaking changes in Zola 0.19. Let’s update and push the Netlify config to see the results.

🐢 We need to add language support for Nushell to prevent warnings. Take a look at how to add a syntax file to Zola.

🐇 I added

extra_syntaxes_and_themes = ["syntaxes"]

to the [markdown] section and added a syntaxes/nushell.sublime-syntax file copied from https://github.com/kurokirasama/nushell_sublime_syntax. I’m getting:

Error: Reason: Error while compiling regex '\b(?x: 7z | ?
...
s-to-gdrive | usage | ver | verify | weather | wget-all | which-cd | wifi-info | wifi-pass | xls2csv | ydx | yt-api | ytcli | ytm | z | zi)\b'
Oniguruma error: target of repeat operator is not specified

🐢 The syntax highlighter may be a bit buggy. Let’s try to fix this if it’s a one-off.

🐇 Found the bug. There is a ? in the regex that causes it to fail. Now it compiles, and Nushell blocks are colored.

🐢 Cool. Let’s fix the other warnings now. shell and console are not recognized, it looks like.

🐇 There is only Bash listed in https://www.getzola.org/documentation/content/syntax-highlighting/.

🦁 What do you think about migrating to mdBook? We already maintain mdBook for XVC; what about just moving the site to mdBook?

🐢 I thought about this before, and the only downside is the lack of an RSS feed.

🦊 I found this: https://github.com/theowenyoung/mdbook-rss

🐢 Now, this changes everything. We can even move the nedriy.at site to mdBook in this case.

🐇 Then, let’s start working on this. The site will be a technical book site in this case.

🐢 Does mdBook support Nushell syntax?

🐇 Nushell is not in the listed languages. mdBook uses highlight.js, and in its listed languages, we don’t find Nu either.

🐢 We already added Nu support to Zola, and we can just change the theme. This is a blocker in my opinion.

🐇 I searched for Nushell highlight.js support, and nothing appears. I think we can just postpone until Nu has more support on this front.

🐢 Yes. Let’s first try this change in the non-technical blog, and we can come back to this issue. Now, we’ll update console and shell to bash, I think.

🐇 Replaced shell and console with bash.

🐢 There is a file for ggplot that has warnings from earlier incarnations. We also lack syntax highlighters for Vim and Tmux.

🐇 There is a sublime-syntax file for Tmux at https://raw.githubusercontent.com/gerardroche/sublime-tmux/refs/heads/master/Tmux.sublime-syntax, but do we need it for a single file?

🐢 Let’s set it to plain text.

🐇 Now we only have Mermaid warnings left.

🐢 There should be a diagram at https://emresahin.net/developing-a-gitignore-crate/, but it doesn’t show up. We need a shortcode to show these, like the YouTube shortcode. Now let’s get back to theme selection.

🐇 I tested Karzok, but it doesn’t have category and tags support.

🐢 And I tested https://github.com/micahkepe/radion, but the best so far is the apollo theme. I’m struggling to modify the index page, though. I forgot that I modified the theme’s index.html file. I have content in /content/_index.md and a modified index.html in /themes/anemone/templates/index.html to show the content, tags, categories, etc. It should be fixed now.

🐇 Ah, cool. Can we clean the recent duplicate pages now?

🐢 Yeah, let’s take a look.

🐇 What about adding at least command completions for Nushell?

🐢 Let’s read a bit about clap_complete_nushell.

🦊 There seems to be a nu-complete command. Let’s check its documentation.

🐇 Nothing was found, and Kagi doesn’t help much either.

🐢 There is a completions document for Nushell: https://www.nushell.sh/book/custom_completions.html

🐇 There is a tool called carapace to provide completions across shells.

🐢 Its documentation is thin, and I’m not sure if it supports dynamic completions out of the box. I believe instead of adding a carapace setup, we can just write a Nushell completion script that will use JSON output from the commands and add some (maybe hidden) utility commands to support it.

🐇 There are a set of example scripts in the Nushell repo: https://github.com/nushell/nu_scripts/tree/main/custom-completions

🐢 The reason I want to write custom completions for Nushell is that it will be an exercise for the scripting language. gh completions are not as scary as a Bash script.

🐇 git completions are a better example for XVC. They simply run git whenever necessary. We can start from a static completions command and update this with dynamic completions manually. It will teach a lot.

🐢 I forked the nu_scripts repo and will add XVC completions script there.

🐇 Then let’s begin by adding Nushell static completions. Shall we add a command for this?

🦊 Reviving the completion command we removed in 0.6.13?

🐢 We shouldn’t list it. We can make a _comp subcommand for the time being and generate and distribute completions in the repository. When clap_complete_nushell has the feature parity to provide dynamic completions, we can remove these commands.

🐇 What will we use this for other than generating completions?

🐢 Maybe dynamic completions can call this as well.

🐇 Added Nushell static completions to be output using xvc _comp generate-nushell. Let’s bump up the version to 0.6.15.

cargo set-version 0.6.15-alpha.1
   Upgrading xvc from 0.6.14 to 0.6.15-alpha.1
...

🐢 I noticed we forgot a line in the CLI command handler that asserts xvc_root_opt.is_some(), and this fails when we run xvc outside of repositories. We need to release this version quickly.

🐇 Oops, now, ok, let’s write a static Nushell generator and just release quickly.

🦊 Generating completions with

xvc comp generate-nushell

🐢 Completion command is run with comp instead of _comp. Should we rename it?

🐇 Renamed it to _comp. It’s not hidden, but at least we can be sure that it won’t be misunderstood as a common command.

🐢 Bumping the version again. Now let’s source the generated script and test it.

cargo set-version 0.6.15-alpha.2
   Upgrading xvc from 0.6.15-alpha.1 to 0.6.15-alpha.2
...

🦊 Yep, it works. We now have completions for Nushell.

🐢 Let’s update the completions documentation.

🐇 Done. Now, let’s take a look at CI and see what fails.

ghrl | first
╭──────────────┬─────────────────────────────────────────────────────────╮
│ conclusion   │ success                                                 │
│ displayTitle │ Add Nushell completions                                 │
│ headBranch   │ nushell-completions                                     │
│ url          │ https://github.com/iesahin/xvc/actions/runs/13070200765 │
╰──────────────┴─────────────────────────────────────────────────────────╯

🐢 It fails because of coverage, not the tests. Codecov says the new code isn’t tested.

🐇 The added xvc _comp command isn’t tested. We can add a test running those lines and testing if the command outputs a completion script.

🐢 We have a test for completions. We can add a test that runs the lines.

🐇 Added a test and bumping up the version.

cargo set-version 0.6.15-alpha.3
   Upgrading xvc from 0.6.15-alpha.2 to 0.6.15-alpha.3
...

🦊 We can add some more coverage while waiting for the tests.

🐇 XvcOutputLine implementation seems to have no tests. It’s weird because we use these everywhere.

🐢 I’m not sure we use this particular implementation; we just use XvcOutputLine::Info(s), not XvcOutputLine::info(s) anywhere. We can delete these methods actually.

🐇 We’ll add JSON output via this particular struct. Can we refactor these to use formatting for JSON, for example? Or use these to output JSON?

🦊 We can add a formatter to XvcOutputLine to output structures.

🐢 The enum is now defined as:

#[derive(Clone, Debug)]
pub enum XvcOutputLine {
    /// The output that we should be reporting to user
    Output(String),
    /// For informational messages
    Info(String),
    /// For debug output to show the internals of Xvc
    Debug(String),
    /// Warnings that are against some usual workflows
    Warn(String),
    /// Errors that interrupts a workflow but may be recoverable
    Error(String),
    /// Panics that interrupts the workflow and ends the program
    /// Note that this doesn't call panic! automatically
    Panic(String),
    /// Progress bar ticks.
    /// Self::Info is also used for Tick(1)
    Tick(usize),
}

Here, these fields can also have a formatter that will render the string in a particular format. For example, the output can be

XvcOutputLine::Output(XvcJsonFormatter, String)

🐇 I’m not sure this is a good idea. Output already specifies this string as output. We can have a wrapper instead, like,

struct XvcJsonOutput(Format<XvcStructuredOutput>, XvcStructuredOutput)

and we can use the supplied format to render XvcStructuredOutput to an output line with XvcOutputLine::Output. If we don’t provide output as structured, it will be too much error-prone work to convert the current outputs to structured.

🦊 The transition will also be gradual. We may not need structured output for most of the commands. We can start with xvc file list and convert others as we go.

🐢 This is sensible. By the way, coverage still didn’t increase. There may be something going on with Codecov or running the test.

ghrl | first
╭──────────────┬─────────────────────────────────────────────────────────╮
│ conclusion   │ success                                                 │
│ displayTitle │ Add Nushell completions                                 │
│ headBranch   │ nushell-completions                                     │
│ url          │ https://github.com/iesahin/xvc/actions/runs/13087431038 │
╰──────────────┴─────────────────────────────────────────────────────────╯

🐇 Let’s run the test:

cargo test -p xvc --test test_completions
...
test test_completions ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.80s

🐢 Can we make sure the output is a Nushell script and not an error message?

🐇 Let’s print it out.

🐢 It looks like when the COMPLETE environment variable is set, it never calls _comp subcommand and never calls those lines.

cargo set-version 0.6.15-alpha.4
   Upgrading xvc from 0.6.15-alpha.3 to 0.6.15-alpha.4
...

🐢 Let’s make a release for 0.6.15. Coverage is OK now.

cargo set-version 0.6.15
   Upgrading xvc from 0.6.15-alpha.4 to 0.6.15
...

gh pr merge --squash --body $"(open CHANGELOG.md | lines | skip 2 | take 5)" --subject "Add static nushell completions"

🐇 Merged the PR.

🐢 Releases should appear in a few minutes.

🐇 We need to tag the merge commit for this.

🐢 Oh, yep. AFAIK Lazygit doesn’t have something for git push --tags. Let’s push from the CLI.

git push --tags
You are on the main branch. Skipping CHANGELOG.md check.
To github.com:iesahin/xvc
 * [new tag]         v0.6.15 -> v0.6.15

🦊 These commands, especially tables, are not rendered correctly on the web. We need to change the theme, I think.

# List issues related to me
export def ghil [] { 
  (gh issue list --search "involves:iesahin" --json title,url) | from json
}

# Add a comment to an issue
export alias ghic = gh issue comment

# Create an issue
export alias ghiC = gh issue create $in

# View an issue
export alias ghiv = gh issue view

# View an issue in the browser
export alias ghiw = gh issue view --web

# List PRs in a table
export def ghpl [] {
  (gh pr list --json "title,url,headRefName" ) | from json
}

# View a PR
export alias ghpv = gh pr view

# View a PR in GitHub
export alias ghpw = gh pr view --web

# Checkout a PR
export alias ghco = gh pr checkout

# Add a comment to a PR
export alias ghpc = gh pr comment

# Show changes in a PR
export alias ghpd = gh pr diff

# Add a review to a PR
export alias ghpR = gh pr review

# Create a PR
export alias ghpC = gh pr create

# Merge a PR to main
export alias ghpM = gh pr merge

# List GH CI runs in a table
export def ghrl [] {
  (gh run list --json conclusion,displayTitle,headBranch,url ) | from json 
}

# View a GH CI run
export alias ghrv = gh run view

# View a GH CI run in the browser
export alias ghrw = gh run view --web 

# View output from a failed run
export alias ghrf = gh run view --log-failed

# Search code and get results in a table
export def ghsc [] { 
  (gh search code $in --json repository,path,textMatches,url) | from json
}

Aliases don’t allow pipes in Nushell, so I used def for those that require them.

When there is an $in parameter in an alias, it expects the input from a pipe. An issue can be created like this, for example:

"Your software sucks" | ghiC

To search for the code copied to the clipboard and get only the URLs, you can use:

pbpaste | ghsc | get url

To open them in different browser windows:

pbpaste | ghsc | get url | each { |u| start $u }

I’ll add a shortcut key for this last one. It’s so quick! :D

🐇 We can just complete the completions for this version and add JSON output for all commands to use in nushell. In the meantime, dynamic completions support for nushell is likely to be added.

🐢 Umm, yep. Then we’ll continue to run tests with zsh for the current version.

🐇 run-tests is a zsh file, and we will continue to use it. We can switch to nushell eventually as we add more compatibility with it, but let’s not do this now.

🐢 Where were we the last time for completions?

🐇 We added xvc_path_completers. Let’s check the TODO list now.

🐢 We still need tracked_targets completions in a few places.

🐇 Added those.

🐢 Now we have some strum completers. Let’s finish these up as well.

🐇 Ok. They are done as well.

🐢 Now, let’s start adding a storage_identifier completer. It should read all XvcStorage records and list their names.

🐇 Added a storage_identifier completer. Let’s add it to all places where we use storage_identifiers.

🐢 Now, we need completers for pipeline and step names. These are straightforward.

🐇 Added these. Let’s fill up where they are needed.

🐢 I want to skip some of the fine-grained completions in this version. We can have a specific completer for --params options for files and params inside, for example. Or a special completer for directories.

🐇 Let’s not forget these. Reading YAML files and extracting hyperparameter keys for the prompt would be a really good feature for the user.

🐢 I agree. These are good features in general, but we need to ship the current version as soon as possible.

🐇 Let’s check the TODO comments once more.

$ rg 'TODO:' 
...
- ✅ pipeline/src/pipeline/api/update.rs:    /// TODO: Add a repository_dirs completer (11:08)
- ✅ file/src/remove/mod.rs:    /// TODO: Add a storage_identifier completer (11:08)
- ✅ file/src/bring/mod.rs:    /// TODO: Add a storage_identifier completer (11:08)
...

🦊 We can use xvc_path_completer for the tracked directory completer for the time being. For pipeline update, we can use a ValueHint instead. It’s not necessary to use a custom completion.

🐢 For the xvc file copy destination, we can have a file or a directory that we track and is not available, or we don’t track and is available. It’s a similar situation to xvc_path_completer, but we also need to check the local paths. It requires some more care.

🐇 Now we can begin to run the tests.

🐢 Is this building?

🐇 It should.

🐢 We also have one bug. xvc shouldn’t print help text when the COMPLETE environment variable is set.

🐇 Umm, yeah, that’s a blocker.

🐢 Fixed it. We can bump the version and push the changes.