🐇 Starting with completions is better. One question I have is whether the completions work for subcommands as usual, like in the static completions.

🐢 Yes, they work. We need to add:

source <(COMPLETE=zsh xvc)

to .zshrc, though.

🐇 It looks like we can drop the xvc completions command. We’re just checking the COMPLETE environment variable, and there’s no need for a separate command in this case.

🐢 Yes, let’s remove that.

🐇 Maybe we can employ a completions module for all completion-related functionality. It’s a separate thing, you know.

🐢 Which completions do we need?

🐇 We can just mark them with TODOs now.

🐢 Yes, let’s check where we need completions and what.

🐇 I noticed we’re repeating options in XvcFileCLI, and some of the options are missing, e.g., --from-ref and --to-branch.

🐢 Because it can compile into a different binary, and global options don’t work that way. Maybe we can move all these binaries into different files under lib. We can have feature flags to turn off certain features and compile different binaries.

🦊 Let’s search for it; I haven’t seen it before: building different binaries with feature flags using cargo.

🐢 There is no clear-cut solution, but it looks like we can move all binaries to the root with the required-features field.

🐲 We can postpone this to another release.

🐇 Yes, let’s not spend time on this now.

🐢 Should we try making pipeline_name a global option? It’s repeating everywhere.

🐇 That will make it easier to maintain.

🐢 We’ll have to pass pipeline_name to subcommands, though.

🐇 I think we can have a set of global options that we can pass. For the time being, that’s only the pipeline_name.

🐢 Okay. Let’s do this.

🐇 A similar option is the step name for pipeline steps.

🐢 Dependencies will need a revamp in the next version anyway. So let’s keep it for now.

🐲 Also, the semantics of step-name are different for these commands. step new interprets it as a new name, while step dependency interprets it as an existing name. The first can be renamed to --name, and the second can be --to, as one of its aliases suggests.

🐢 Yes, let’s keep it for now, and we’ll continue to work on others.

🦊 I want to post a comment in the clap discussions.

🐢 Wrote it. Now let’s build it after changing the pipeline_name.

🐇 It compiles. Should we test it?

🐢 I think we’ll test after all this completion work is done.

🐇 We did most of the strum-related completion.

🐢 Didn’t test them yet, though.

🐇 This is Rust. It will work if it compiles, and it compiles.

🐇 The examples are only found in the tests: https://github.com/clap-rs/clap/blob/master/clap_complete/tests/testsuite/engine.rs#L604

and we can try to create those random strings this way.

🦊 The issue is that we may not be able to use the derive method to add these custom completers. All examples are using the builder API.

🐢 We should be able to obtain the end result from derives and modify them to use these custom completers, but let’s try to use add = ArgValueCompleter first.

🐇 It looks like we have another problem. We need to add a rust-toolchain.toml file to the project, you know, to keep it on the stable channel.

🐢 Ah, yeah, we now use multiple channels. Let’s do that.

🐇 I found an example, actually:

#[derive(Debug, Parser)]
struct Cli {
    #[arg(long, add = ArgValueCompleter::new(custom_completer))]
    custom: Option<String>,
}

🐢 I’m trying to add this to the --from-ref option of Xvc. It’s not running anything. The custom completer should return a random string, but it doesn’t.

🦊 Maybe test with a constant first.

🐢 It didn’t work that way either.

🐇 Let’s do a cargo clean.

🐢 Nothing has changed.

🐇 Let’s take a look at the output of xvc completions:

xvc completions
...
'(--skip-git)--from-ref=[Checkout the given Git reference (branch, tag, commit etc.) before performing the Xvc operation. This runs \`git checkout <given-value>\` before running the command]:FROM_REF:_default' \
...

🐢 This is the only place --from-ref is mentioned and, as far as I can see, there is nothing that calls a dynamic command here.

🐇 Umm, right. There’s something weird here. Maybe we lack the ArgExt trait or something.

🐢 It should be turned on by clap-complete with its unstable-dynamic feature, but who knows.

🐇 It didn’t work.

🐢 ArgExt is available, but it isn’t being used.

🐇 I think I found the answer in https://jj-vcs.github.io/jj/latest/install-and-setup/#command-line-completion:

source <(COMPLETE=zsh xvc)

is the command we should use.

🐢 It doesn’t produce a completion script, though.

🐇 We have this in jj/cli/src/cli_util.rs:

        if env::var_os("COMPLETE").is_some() {
            return handle_shell_completion(ui, &self.app, &config, &cwd);
        }

🐢 I think the whole handle_shell_completion set is written by them. I’d like to see what jj with COMPLETE=zsh outputs.

🐇 Building it to see now.

🐢 As expected, it calls a function:

❯ COMPLETE=zsh target/debug/jj
#compdef jj
function _clap_dynamic_completer_jj() {
    local _CLAP_COMPLETE_INDEX=$(expr $CURRENT - 1)
    local _CLAP_IFS=$'\n'

    local completions=("${(@f)$( \
        _CLAP_IFS="$_CLAP_IFS" \
        _CLAP_COMPLETE_INDEX="$_CLAP_COMPLETE_INDEX" \
        COMPLETE="zsh" \
        /Users/iex/github.com/etc/jj/target/debug/jj -- ${words} 2>/dev/null \
    )}")

    if [[ -n $completions ]]; then
        _describe 'values' completions
    fi
}

compdef _clap_dynamic_completer_jj jj

🐇 Now we should make it the same; the shell must call xvc to get the completions. That’s how all these will work.

🐢 Our goal now is to make a random string output from the --from-ref completion.

🐇 The plan is to make completions work as quickly as possible.

🐢 We have done it. 🎉🥳

🐇 Cool. Now we need to fill up all those completion methods.

🦊 Yeah. We can start with basic ones and move from there.

🐢 There will probably be architectural changes as well. We cannot just run xvc functions directly. We need to run internal functions, but not through commands. At that point, when the user hits tab, we don’t know which command to run.

🐇 Maybe it’s time to move to a Git library. “What is the best Git library for Rust?”

🐲 Should we use a Git library?

🐢 It looks like Gitoxide is the default Rust way to interact with Git repositories. We can keep our way of using the Git binary and use this as an experimental way to learn.

🦊 Yep. That’s a good idea. We can include it in the lib for the time being and start to use it in completions.

At work, we use Doppler to manage our secrets. It injects environment variables before running commands. I thought, why can’t I do the same with a simple script?

The following script evaluates all password files with env in their names. I keep my environment variables in files named env-aws, etc., in the following format:

export AWS_ACCESS_KEY_ID="123456...."

The following script, which I called rws (short for run-with-secrets), allows me to run a command like rws s3cmd and use pass to inject the variables. So far, I’m happy with it.

#!/bin/zsh

cmd="$@"

fd -F env $HOME/.password-store | while read file ; do
  bb="${file:t:r}"
  eval $(pass show ${bb})
done

exec $cmd

This script merges all windows from all Tmux sessions into a single session. You must specify the target session as a parameter. If you save the script as merge-tmux-sessions.zsh, you can run it like:

merge-tmux-sessions.zsh my-current-session

It will then move all windows into the session you specify.

#!/bin/zsh

# set -vuex
#
#  Specify the target session

target_session=$1

# Get the list of all sessions

sessions=$(tmux list-sessions -F '#S')

# Iterate through each session

for session in ${(@f)sessions}; do
    if [ "$session" != "$target_session" ]; then
        # Get the list of windows in the current session
        windows=$(tmux list-windows -t $session -F '#I')

        # Move each window to the target session
        for window in ${(@f)windows}; do
            tmux move-window -s ${session}:${window} -t ${target_session}
        done
    fi

done

I needed a way to quickly open URLs in the Kagi Summarizer from my terminal. I wrote a small shell function to do that:

ksum() {
  local urlenc=$(jq -rn --arg x "$1" '$x|@uri')
  open "https://kagi.com/summarizer/index.html?target_language=&summary=takeaway&url=${urlenc}"
}

Nowadays, I’m using rclone. It has become my primary way of using Drive on Linux after that half-baked sync application. I discovered that rclone supports server-side MD5 hashes for files. I decided to write a script to delete the duplicate files in Drive.

First, I obtain the MD5 hashes of all files in Drive using:

$ rclone md5hash drive:/ > $HOME/Google-Drive-md5-$(date +%F).txt

This may take some time depending on the number of files, but it finished more quickly than I expected.

The file contents look like this:

39044094333de4a47d7478227cfa22a9  Facebin/vgg-clean/clea_duvall/00000501.jpg
398d44f0b32d1b4855e838754c2c49fc  Facebin/vgg-clean/Bob_Barker/00000445.jpg
c2090f9412b93d71fed884bb45b26518  Facebin/vgg-clean/Adam_Goldberg/00000432.jpg
cd6971f809a534ab02ee1b03eb6c1183  CHECK Uploads/Google Photos/2017/06/IMG_1461.JPG
1fb80cc9ef622410557cb42c4abf26a8  Facebin/vgg-clean/Angell_Conwell/00000510.jpg
160793fd8f24fcf27efb9a2b3698a9c8  Facebin/makimface-artifacts/dataset-images/user-test-v4/00056--/img-5c1fdc5d4fd82c1e12a7d49937d7f47b.png
bcddf6ebbc1eed83950d908c2ccbac4a  Facebin/vgg-clean/danny_pino/00000102.jpg
eb2746a9559e7a93fd0002f4bfd90517  Facebin/makimface-artifacts/dataset-images/user-test/00009--/ds-767cd16bfdf4e31860d597708a586979.png
9605d69aaf5fb83d309f10f9e2544630  Facebin/vgg-clean/Adam_Beach/00000953.jpg
68e6d3ec3cca8a728a78fb60c9a1ddba  Facebin/vgg-clean/corey_stoll/00000137.jpg

The first 32 characters are the MD5 hash of the file, and the rest of the line is the file path.

You’ll probably see some blanks for the MD5 hashes of certain files. It’s important to remove these from the list, as these are your Google Docs, spreadsheets, etc.

$ grep '[^0-9a-f]' $HOME/Google-Drive-md5-$(date +%F).txt > $HOME/Google-Drive-md5-cleaned.txt

Next, we sort the file using:

$ sort $HOME/Google-Drive-md5-cleaned.txt > $HOME/Google-Drive-md5-sorted.txt

We keep all intermediate files because you might want to review the differences after the cleanup.

Then, we write the following script and run it:

#!/bin/zsh

PREV_MD5=""
PREV_PATH=""
CURRENT_MD5=""
CURRENT_PATH=""
MD5_FILE=$HOME/Google-Drive-md5-sorted.txt
cat $MD5_FILE | while read current_line ; do
    # echo $current_line
    CURRENT_MD5=$(echo "$current_line" | cut -c -32)
    CURRENT_PATH=$(echo "$current_line" | cut -c 35-)
    # echo $CURRENT_MD5
    # echo "$CURRENT_PATH"
    if [[ "$CURRENT_MD5" == "$PREV_MD5" ]] ; then
        echo "EQUAL: $CURRENT_MD5 $PREV_MD5"
        echo "DELETE: drive:/$CURRENT_PATH"
        rclone -v delete "drive:/$CURRENT_PATH"
    else
        PREV_MD5=$CURRENT_MD5
        PREV_PATH=$CURRENT_PATH
    fi
done

You can save this script as google-drive-delete-duplicates.sh and run:

$ chmod +x google-drive-delete-duplicates.sh
$ ./google-drive-delete-duplicates.sh

The script checks each line one by one; if two duplicate MD5 hashes are found consecutively, the second file is deleted. It keeps only one of the duplicates, even if there are more than two copies.

I have gained about 200GB by running this script.

  In[1]: 10 + 10

 Out[1]: 20.0


           _____
  In[2]: ╲╱ 100

 Out[2]: 10.0


             ┌                  ┐
  In[3]: Max │ 10 , 1 , 21 , -3 │
             └                  ┘

 Out[3]: 21.0


  In[4]: ⟨Emre's Number⟩ ≡  79

 Out[4]: 79.0


           _______________
  In[5]: ╲╱ Emre's Number

 Out[5]: 8.888194417315589


                      2
  In[6]: Emre's Number

 Out[6]: 6241.0


                      Emre's Number
  In[7]: Emre's Number

 Out[7]: 8.1759873707105095e149

It looks a bit heavy for a CLI calculator, as it is written in Haskell and downloads 100+ MB of libraries, but when you need ASCII art to display your calculations or want to use spaces in variable names, it may prove useful.

• I began to use Intention to limit my Twitter time. It allows you to set a limited time (1, 5, 10, or 15 minutes) for yourself and tracks the total time you spend on addictive sites. When this total time is lower than your goal for a period, you get a streak. It looks visually and psychologically nicer than LeechBlock.

• I read the git reset section in the Git book. It details how git reset behaves with its --soft, --mixed, and --hard parameters. The first resets only the HEAD; the second resets both the index and HEAD; and the third resets the working tree and copies files back from the current HEAD to the working tree.

  One important point: Contrasting git checkout master and git reset master: the first moves HEAD to the master branch, while the second moves the current branch to master.

  git reset can also be used to squash commits. Basically, you git reset --mixed to an earlier commit like HEAD~3 and recommit. This creates a new commit, taking HEAD~3 as the parent and skipping HEAD~2 and HEAD~1, resulting in a new HEAD.

• Here are many useful pure Bash functions to be used in scripts. I’m a Zsh person, but writing Bash scripts is more portable, of course.

I have a shell script to add daily recurring tasks, such as “Drink Water” or “Pray Maghreb,” to this file. SimpleTasks expects the file to be at $HOME/Dropbox/todo/todo.txt, and I use a template file, such as template.txt, to populate it. However, as anyone who has used TODO files knows, there are always unfinished tasks; for instance, I don’t always drink my daily allotted amount of water.

So, my evening script moves completed tasks (marked as DONE) to their own file, removes recurring tasks from todo.txt, and repopulates them for a new day. (Note: The code below is not usable as-is; you must define the TODO, DONE, and TEMPLATE file paths correctly.)

TODO=.../todo.txt
DONE=.../$(date +%F)-done.txt
TEMPLATE=.../template.txt

#Move Done to $DONE
grep -wE '^x ' $TODO >> $DONE
sed -i '/^x /d' $TODO

#Delete lines with @Evening, @Morning or @Regular tags from todo
sed -i "/\b\(@Morning|@Evening|@Regular\)\b/d" $TODO
cat $TODO_TEMPLATE >> $TODO

I have configured cron to run this every day at 6 PM. (My “evening” starts earlier than my morning.)

Bir dizindeki tüm dosyalar: ls *

Bulunduğumuz dizin ve tüm alt dizinlerdeki dosyalar: ls **/*

Son iki haftada değiştirilmiş dosyalar: ls **/*(.mw-2)

Boyutu 100 MB’tan büyük olan dosyalar: ls **/*(.Lm+100)

Sadece okunabilir olan dosyalar: ls **/*(.R)

/etc dizininde dünya tarafından yazılabilir olan dosyalar: ls /etc/**/*(.W)

/etc dizininde son bir haftada değiştirilmiş ve dünya tarafından yazılabilir olan dosyalar: ls /etc/**/*(.Wmw-1)

zsh’in sağladığı bu gibi seçeneklerin yanında dosya adını parçalamak da kolay. Örneğin dosyanın uzantıdan önceki kısmını almak için *(:r) yazıyoruz. Dizindeki jpg dosyalarını ImageMagick’in convert programıyla aynı isimli png dosyalarına çevirmek şöyle mümkün:

for f in **/*.jpg(.) ; do
   convert ${f:r}.jpg ${f:r}.png
done

Burada anlatılanları find yoluyla yapmak da mümkün ancak çok daha uzun sürebiliyor. Bilhassa dosya adlarını parçalamak, dönüştürmek gibi işlemleri yaparken zsh’in nimetleri çok. Tek satırlık bir operasyonla çok güçlü bazı dönüşümler yapabilmek mümkün.

I have used ~ several times in scripts, but it may not always expand as expected depending on the environment. Use $HOME to refer to the home directory; it is much more reliable and always works.