Ubuntu Terminal Smarts

I think using the Ubuntu Terminal in a smart way is crucial for day-to-day productivity.

I came across this nice link where several tricks are listed down and explained nicely.

https://help.ubuntu.com/community/UsingTheTerminal

Note: The consistent Copy-Paste mechanism  I have found working across Unity terminal, Xfce terminal, unity gvim, xfce gvim etc is  using:

  • Control + Insert : to copy
    • this is also same as ‘copy on select’ i.e. just select the text and it will be copied automatically.
  • Shift + Insert : to paste

Check it out!

Java, Maven, Scala, SBT Concepts

Am toe dipping into maven. Trying to make sense of how maven fits in with  IDE, command line maven, POM files blah blah etc.

tips:

  • intellij will have default support within the IDE for both Maven and SBT.  So as long as we are not using mvn and sbt  from the command line we should be good.

java fundamental concepts:

maven:

base scala in intellij:

scala with SBT:

5 Skills.

One person recently pointed out to me that a strong ML team needs 5 skills.

I was glad he pointed it out so clearly. Sometimes its easy to know things, but hearing it from someone else and crystallizing it helps a lot.

The 5 skills (in no particular order) are:

  1. Research
  2. Engineering
  3. Data Science
  4. Program Management
  5. Systems Architecture.

The other interesting thing he mentioned which I really liked was about prioritization. Nowadays there is so much of cool technology out there that it becomes super important to do prioritization. This would give one a strong sense of direction.

Tool Shortcuts

As I keep using different tools, it has become necessary to note down the different shortcuts across the tools.

Spyder:

  • (control + 1)  : comment/uncomment lines
  • (control + d)  : delete lines
  • is there a way to cut lines in spyder ?  (besides the standard select and cut technique)

Notepad++

  • (control + q)  : comment/uncomment lines for different file formats e.g. .sh / .bat

GVim

  • yy  : cut lines
  • pp : paste lines

VS:

  • (control + x) : cut lines
  • (control + kd) : format correction

IntelliJ IDEA:

  • (alt + F12) : command prompt
  • (control + alt + l) : format correction
  • (control + d) : move back from sbt’s scala console (‘sbt’ -> ‘console’) back to the sbt console.
  • (control + ‘/’) : comment / uncomment

 

 

References:

 

 

Git : Undo commits, Viewing Unpushed Git Commits

Tips:

[1] First note that there is a difference between ‘revert’ and ‘reset’

 

[2]  Undoing a ‘git add’ before doing a ‘git commit’

Note:  For this scenario, most of the above links are recommending doing a ‘git reset’. There is a difference between a ‘soft’ reset and a ‘hard’ reset.

 

[3] Recently I had to undo a commit I had made into my local git repo. I was under the impression that reverting commits in Git is a big hassle. To my surprise, it didn’t prove to be too much of a pain. These are things I had to do basically:

  • couple of git reset --soft HEAD~ commands (because there was stuff I had pulled into my repo after commiting)
  • couple of   git checkout —  <file> commands to undo my inadvertent changes.

That was it!  I got my repo back to sanity.

 

[4]  Another import task is to view unpushed git commits. this is an important thing to do, especially before making commits into a remote branch.

 

References:

 

 

Git : Syncing a Fork

1. Git Diff

Suppose git status looks like this.

gitstatus

  • git diff --staged will only show changes to files in the “staged” area.
    • So only changes to data.json  and /python/amlLatency.py  will be shown
  • git diff HEAD will show all changes to tracked files.
    • So changes to data.json,  /python/amlLatency.py  and /python/retrieveStreamingDataRedis.py will be shown
  • git diff
    • Only changes to python/retrieveStreamingDataRedis.py will be shown

2. Syncing  a Fork  – from GitHub.

  1. Open your fork repo on GitHub.
  2. Click on Pull Requests.
  3. Click on New Pull Request. By default, GitHub will compare the original with your fork, and there shouldn’t be nothing to compare if you didn’t make any changes.
  4. Click on switching the base (if no changes were made in the fork) or click Edit and switch the base manually. Now GitHub will compare your fork with the original, and you should see all the latest changes.
  5. Click on Create to create a pull request for this comparison and assign a predictable name to your pull request (e.g., Update from original).
  6. Click on Send pull request.
  7. Scroll down and click Merge pull request and finally Confirm merge (If your fork didn’t have any changes, you will be able to merge it automatically).

3. Syncing  a Fork  – from GitHub.

  • View existing remotes

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git remote -v
multiworldtesting https://github.com/multiworldtesting/decision.git (fetch)
multiworldtesting https://github.com/multiworldtesting/decision.git (push)
origin https://github.com/abgoswam/decision.git (fetch)
origin https://github.com/abgoswam/decision.git (push)

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git fetch multiworldtesting –> Interestingly this didn’t do anything for me. The reason being there were already local branches for ‘remotes/multiworldtesting/*’.  I went ahead and added another remote called ‘upstream’

  • Setting up a new remote (in this example not really needed, but setting up new remote still)

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git remote add upstream https://github.com/multiworldtesting/decision

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git fetch upstream
From https://github.com/multiworldtesting/decision
* [new branch] Declarative -> upstream/Declarative
* [new branch] demo -> upstream/demo

  • View remotes

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git remote -v
multiworldtesting https://github.com/multiworldtesting/decision.git (fetch)
multiworldtesting https://github.com/multiworldtesting/decision.git (push)
origin https://github.com/abgoswam/decision.git (fetch)
origin https://github.com/abgoswam/decision.git (push)
upstream https://github.com/multiworldtesting/decision (fetch)
upstream https://github.com/multiworldtesting/decision (push)

  • View branches

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git branch -va
* master 71a8dc2 fixed bug with asa due to lack of timestamp and ids on outcome and reward objects
remotes/multiworldtesting/Declarative f08fc46 MSN meeting

  • Sync with upstream (and fail): 

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git checkout master
M ClientDecisionServiceSample/Program.cs
M ClientDecisionServiceSample/SingleActionSamples.cs
Already on ‘master’
Your branch is up-to-date with ‘origin/master’.

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git merge upstream/master
Updating 71a8dc2..1175689
error: Your local changes to the following files would be overwritten by merge:
ClientDecisionServiceSample/Program.cs
Please, commit your changes or stash them before you can merge.
error: Your local changes to the following files would be overwritten by merge:
ClientDecisionServiceSample/SingleActionSamples.cs
Please, commit your changes or stash them before you can merge.
Aborting

  • Note how this failed because there were 2 files in my origin/master which are being edited.

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git status
On branch master
Your branch is up-to-date with ‘origin/master’.
Changes to be committed:
(use “git reset HEAD <file>…” to unstage)

modified: ClientDecisionServiceSample/Program.cs

Changes not staged for commit:
(use “git add <file>…” to update what will be committed)
(use “git checkout — <file>…” to discard changes in working directory)

modified: ClientDecisionServiceSample/SingleActionSamples.cs

F:\decision [master +0 ~1 -0 | +0 ~1 -0]> git stash
Saved working directory and index state WIP on master: 71a8dc2 fixed bug with asa due to lack of timestamp and ids on outcome and reward objects
HEAD is now at 71a8dc2 fixed bug with asa due to lack of timestamp and ids on outcome and reward objects

  • Verify and sync with upstream finally: 

F:\decision [master]> git status
On branch master
Your branch is up-to-date with ‘origin/master’.
nothing to commit, working directory clean

F:\decision [master]> git merge upstream/master
Updating 71a8dc2..1175689
Fast-forward
.gitignore | 3 +
ClientDecisionService/App.config | 2 +-
ClientDecisionService/AzureBlobUpdater.cs | 2 +-

  • Un-stash : 

F:\decision [master]> git stash apply
Auto-merging ClientDecisionServiceSample/SingleActionSamples.cs
Auto-merging ClientDecisionServiceSample/Program.cs
On branch master
Your branch is ahead of ‘origin/master’ by 15 commits.
(use “git push” to publish your local commits)
Changes not staged for commit:
(use “git add <file>…” to update what will be committed)
(use “git checkout — <file>…” to discard changes in working directory)

modified: ClientDecisionServiceSample/Program.cs
modified: ClientDecisionServiceSample/SingleActionSamples.cs

  • Check Status: 

F:\decision [master +0 ~2 -0]> git status
On branch master
Your branch is ahead of ‘origin/master’ by 15 commits.
(use “git push” to publish your local commits)
Changes not staged for commit:
(use “git add <file>…” to update what will be committed)
(use “git checkout — <file>…” to discard changes in working directory)

modified: ClientDecisionServiceSample/Program.cs
modified: ClientDecisionServiceSample/SingleActionSamples.cs

no changes added to commit (use “git add” and/or “git commit -a”)

4. Git checkout – For undoing changes

5. Git Commit History

  • To check the list of commits that have gone in
    • git log
  • To view details of last 4 commits
    • git log -p -4
  • To view changes in a file
    • git log -p .\DecisionServiceWebAPI\packages.config

6. Git Commit History

 

References:

  1. http://stackoverflow.com/questions/292357/what-are-the-differences-between-git-pull-and-git-fetch

 

Sed Usage.

Two pretty useful sed commands:

[1] Show a particular line only

[2] Dealing with range of lines:

[3] Prepend Text to a File at the Command Line

  • sed -i ‘1i id,gender,age’ mldataset.csv  (Prepend Text to a File at the Command Line)

[4] Remove blank lines from file.

  • sed ‘/^$/d’ input.txt > output.txt
  • grep -v ‘^$’ input.txt > output.txt

(Both grep and sed use special pattern ^$ that matchs the blank lines. Grep -v option means print all lines except blank line.)

References: