devlog 13
🐇 So, what’s next?
🐲 We can work on completions or GUI.
🐢 Completions are similar to homebrew work. It’s in another ecosystem and feels boring.
🐲 We can try to make it in Rust: “Writing shell completions in Rust”
🐢 There are two solutions: One is clap_complete, https://docs.rs/clap_complete/latest/clap_complete/ and the other is https://github.com/JoshMcguigan/shell_completion. The latter is very new but may be what we need, writing completions only with Rust.
🐲 shell_completion crate is very bare bones. It doesn’t have anything, actually. https://github.com/JoshMcguigan/shell_completion/issues/1
🐇 The feature that we need is dynamic completion. We’ll write something similar to https://docs.rs/clap_complete/latest/clap_complete/engine/struct.ArgValueCompleter.html for this.
🦊 I think we can just a new branch.
🐲 We have a branch for homebrew, should we merge it?
🐢 I think, yes, we can merge it. We’ll fix if it breaks. It’s a separate workflow file anyway.
🦊 Created the PR.
ghpl 264 add brew tap add-brew-tap OPEN 2025-01-03T05:14:45Z
🐢 Merged it.
🐇 Now let’s create a new branch and add clap-complete to Cargo.
🐲 There are some packages that we need to take a look at. thiserror now has v2.
🐢 Upgraded packages and compiled. It works.
🐲 Let’s update the version.
🐢 Done. Now we can add clap-complete.
🦊 Added with unstable-dynamic command. Can we build again?
🐢 Built it. Now we can add a completions subcommand to Xvc.
🐲 Is this the right name for this?
🐢 It will output a shell script that we can source in the shell.
🐲 Ok, let’s add the subcommand now.
🐢 The example doesn’t work with clap builder.
🐇 Let’s search for it.
🐢 I think we’re extending ourselves a bit when trying to add all dynamic features of completion at once. We can just start adding completion to xvc-test-helper
🐇 Yep, that’s a better idea. We can have much more examples and it’s more straightforward for sure.
🐢 One thing I noticed when working yesterday is that using auto-save was actually preventing much of these swap errors.
🐲 It was a bit slow. Can we take a look at a few others?
🐢 https://github.com/okuuva/auto-save.nvim seems a bit more polished.
🐇 Installed it and didn’t see an effect yet
🐢 We may need to restart this
🐲 Now, we can get into test-helper completions.
🐢 Done. Added these quickly as you thought. Now adding this to main seems much easier.
🦊 That was a nice approach. Let’s dive into adding this to main.
🐇 Should we go with a separate command or just an option
🐲 Our commands have distinct initial letters to use them only with letters. Adding another top level command for this will make c useless.
🐢 I don’t think that’s a valid concern. We can create aliases for all commands and completions shouldn’t have to have an alias. But I agree that completions should not be a top level command. It will be run once for installation at most.
🦊 I agree. Let’s call it –completions. It will be run as xvc --completions zsh and will print out the completions.
🐢 Ok. Let’s do this.
🦊 No errors left. Let’s install this version.
🐢 We get
error: 'xvc' requires a subcommand but one was not provided
[subcommands: file, init, pipeline, storage, root, check-ignore, aliases, help]
Usage: xvc [OPTIONS] <COMMAND>
For more information, try '--help'.
🦊 Umm, ok. I think we don’t have an option to create a command like that. Then we’ll have to print completions with a subcommand.
🐲 The above discussion is now moot. ❌
🐢 We can try to push but probably it’s not worth it.
🐲 Let’s not lose time on this. I think we can remove aliases command and replace it with completions and add single letter aliases to subcommands.
🐢 We can have an option in completions command to print aliases. That will work.
🦊 I don’t think people will use aliases command if we have single letter command aliases.
🐢 You may be right. No need to try to maintain that at this time.
🐢 Good morning and I’m fed up with these blink.cmp errors, you know.
🐇 Reinstalling blink works. It’s interesting to rely on such an unreliable software.
🐢 Oh, yeah. It’s interesting. Where were we yesterday?
🐲 We decided to rename aliases command to completions.
🐇 And waiting for Rust-Analyzer to complete its analysis.
🐢 Uh, yeah. I see.
🦊 The code itself is very short actually.
if let Some(shell) = cli_opts.completions {
let mut cmd = XvcCLI::command();
generate(shell, &mut cmd, "xvc", &mut io::stdout());
return Ok(None);
}
🐲 We’ll use output! macro instead of writing to io::stdout(), right?
🐢 Yes, we can rely on the usual output system. It will be slower to create an output thread but shouldn’t matter to output a shell script.
🐇 It takes a while for rust-analyzer to scan all the directories it looks. When I close the project, it just cannot reload it immediately.
🐢 We can use cargo clean time to time.
🦊 Rust-analyzer finished scanning, let’s try to rename AliasesCLI.
🐲 Let’s keep these here, maybe we’ll need them in our scripts.
# Standard Xvc command aliases for longer commands.
alias xls='xvc file list'
alias pvc='xvc pipeline'
alias fvc='xvc file'
alias xvcf='xvc file'
alias xvcft='xvc file track'
alias xvcfl='xvc file list'
alias xvcfs='xvc file send'
alias xvcfb='xvc file bring'
alias xvcfh='xvc file hash'
alias xvcfco='xvc file checkout'
alias xvcfr='xvc file recheck'
alias xvcp='xvc pipeline'
alias xvcpr='xvc pipeline run'
alias xvcps='xvc pipeline step'
alias xvcpsn='xvc pipeline step new'
alias xvcpsd='xvc pipeline step dependency'
alias xvcpso='xvc pipeline step output'
alias xvcpi='xvc pipeline import'
alias xvcpe='xvc pipeline export'
alias xvcpl='xvc pipeline list'
alias xvcpn='xvc pipeline new'
alias xvcpu='xvc pipeline update'
alias xvcpd='xvc pipeline dag'
alias xvcs='xvc storage'
alias xvcsn='xvc storage new'
alias xvcsl='xvc storage list'
alias xvcsr='xvc storage remove'
🦊 We can use them when adding single letter aliases.
🐇 There is a from_env method for Shell, will we support it?
🦊 I think we can support it. It’s much easier to use to omit the shell.
🐲 We had aliases in xvc-core but we need to access XvcCLI from completions. Completions module must be moved to xvc.
🦊 We added clap_complete dependency to xvc-pipeline and xvc-file crates but I don’t think they are necessary. Let’s remove them now.
🐢 Now only the test helper and xvc crates has clap_complete dependency.
🐇 Are we ready to test?
🐢 Completions are working. 🎉
🐲 Now we need to update the docs and doc tests I believe.
🐢 There are also tests to update.
🐇 Tests are running now. In the meantime can we take a look at the blink configuration?
🐲 When we removed xvc aliases, we also removed pvc, xls and other aliases. Maybe we can add these to the docs.
🐢 We can put them to xvc completions reference for now.
🐢 It takes a while for rust-analyzer to finish analyzing the code base.
🦊 Added aliases for xvc pipeline commands. Do you think we need to repeat root level flags in xvc-pipeline?
🐢 No need to divert attention I believe. Also, I still think there is an easier way to do that.
🐇 Ok. Do you think we should add easier subcommands to step? Like, step new becoming xvc p sn instead of xvc p s n?
🐢 I think we can extend these to even to the toplevel, but shouldn’t make them visible. It will pollute the help text. We can add xvc fl for file list to prevent the space, but hide these from the help text.
🐲 That’s a good idea but it will require to include sub-crate level modules at the top level. We must test it first.
🐢 Let’s finish up the current changes and release them first.
🐇 By the way, we didn’t add two letter abbrevs to xvc storage new subcommands. Do you think we need them?
🐢 When we think about the frequency of these commands, no, I don’t think we need them. Users won’t add a new storage every day.
🐇 By that logic, we shouldn’t need a n for xvc storage new.
🐢 Actually, yeah. maybe we should remove even s.
🦊 storage list may be useful.
🐢 s is a very common letter though. We can have other uses for that letter.
🐇 Updating xvc p s dependency options but it looks making dependency options as subcommands is a better way.
🦊 We didn’t do it because currently we can supply multiple options with a single command. If we go to subcommand route, we’ll have to write all dependencies one by one. Adding multiple subcommands with clap is a bit tricky.
🐢 Umm. yeah, I see.
🐇 Maybe we can provide a separate command to add multiple dependencies. Like, xvc p s d add 'lines=myfile.csv::10-20; file=myimage.jpg; glob=dir/image-10*'
🐲 That looks like the beginning of a language. We need a parser for those strings. They will be freeform.
🦊 Another option is to keep the current options and add commands with the same names.
🐢 That will be confusing. User will have both --param and param and they will work differently.
🐇 We can have an add command that accepts the current options. Parsing will be done by clap just like now, but for a subcommand of dependency.
🐢 The full command will be something like xvc pipeline step --step-name preprocessing dependency add --params 'params.json::batch_size'
🐇 With shorter commnands it’s like xvc p s -s preprocessing d a --params 'params.json::batch_size' and this doesn’t look like ffmpeg monstrocities.
🐲 In any case, this is a backward incompatible change. This should wait v0.7 along with ECS changes.
🐢 ECS changes are not user visible but these are. Certainly needs a minor version update.
🐇 Then, ok, let’s keep the current ones for this version and think about updating them in a future version.
🐢 Ok. Let’s finish up and we’ll create a separate invisible subcommand to ask questions to Xvc repo for completions.
🐇 We completed completions for the common command structure. Now, we need a way to show certain info after certain commands. A tab after xvc p r -p should show pipeline names, for example.
🐢 Umm, yeah. And these should be as quick as possible. They shouldn’t check git or any other things.
🐇 I looked here and there and the only working example I found is here: https://github.com/clap-rs/clap/blob/master/clap_complete/tests/testsuite/zsh.rs#L248 in clap_complete tests.
🐢 Let’s try to dive in. We can start by cloning the repo I believe.