PyTest does allow output to be ‘live printed’
Also, it possible to see logging output in PyTest
Checkout these two links:
Uncategorized
Updating the transformers package
Few steps I follow each time i update the transformers package
git pull pip install --upgrade . pip install -r ./examples/requirements.txt
that’s
Tensorflow GPU windows
conda install tensorflow-gpu
Latex quirks
its interesting how many quirks Latex has.
- I had to use a particular .sty file for the paper.
- to my dismay, I found its not sufficient to simple put it in the same folder as the .tex file
- I had to do the following :
- create directory where my other package files were present
- place the .sty file there
- do texhash
- https://crmda.dept.ku.edu/guides/32.latex_config/32.latex_config.pdf
- for me it was here : C:\Users\agoswami\AppData\Local\Programs\MiKTeX 2.9\tex\latex
- Bibliography
- currently i am doing the following steps. i wonder if this can just be automated without using the command line each time i have to add a citation.
- pdflatex main_trials
- bibtex main_trials
- pdflatex main_trials
- pdflatex main_trials
System Design Interview
The system design interview can be of two kinds:
Object Oriented / Data Model :
- [Course] Programming-foundations-object-oriented-design
- http://www.andiamogo.com/S-OOD.pdf
- https://www.udemy.com/design-patterns-csharp-dotnet/
System Design :
References:
Things you do as a well rounded ML Engineer / Data Scientist
I came across this list somewhere on the web.
I felt the list nicely captures the different kinds of skills a well rounded ML Engineer/Data Scientist develops over his/her career:
|
||||||||||||||||||||||||||||||||||
Python Class Attributes
I recently came across this interesting article on class attributes v/s instance attributes in Python.
References:
- https://www.toptal.com/python/python-class-attributes-an-overly-thorough-guide
- This actually let me to 2 other guides which are very interesting:
- https://pythonconquerstheuniverse.wordpress.com/2012/02/15/mutable-default-arguments/
- https://timothyawiseman.wordpress.com/2012/10/06/class-and-instance-variables-in-python-2-7/
- http://www2.lib.uchicago.edu/keith/courses/python/class/5/##classinst
- This is a very thorough guide. Highly recommended!
Code:
Named Pipes. C# Python. .NET Core
I recently ran into issues using the NamedPipeServerStream API with .NET Core.
- In Windows, NamedPipeServerStream creates a pipe with a defined name in a specific location on the Windows filesystem (\\.pipe\\)
- In a Python client application, we were able to open this pipe for communication using the code snippet : open(r’\\.\pipe\\’ + pipe_name, ‘r+b’, 0)
- However, on Linux, the behavior for the NamedPipeServerStream API is different.
- Looking at the source code for .NET Core, I saw that NamedPipeServer/ClientStream in .NET Core are built on top of Unix domain sockets.
- So, if we want to communicate with a Python client, we have to use Python’s socket module
Code:
- C# (Server) : https://gist.github.com/abgoswam/025bbf7b845259e6cbfb0671c12b51ac
- Python (Client) : https://gist.github.com/abgoswam/7305409388e7192cf7ad2cba72296aab
References:
- https://github.com/dotnet/corefx/commit/fcc5cc9d78b84958394395611ebad9e7be222658
- This is the PR that makes the NamedPipe*Stream API on Unix use Unix domain sockets.
- The description in the commit helped me understand the issue.
- http://jonathonreinhart.blogspot.com/2012/12/named-pipes-between-c-and-python.html
- This is a nice post on C# <-> Python bridge using Named Pipes.
- However, this technique doesn’t work for Linux (when using .NET Core). Hence I am posting this article on how to use Unix Domain sockets instead. 🙂
C# OutOfProcess Python
- WaitForExit Method (Int32):
- true if the associated process has exited; otherwise, false.
- https://msdn.microsoft.com/en-us/library/ty0d8k56(v=vs.110).aspx
- UseShellExecute:
- true if the shell should be used when starting the process; false if the process should be created directly from the executable file. The default is true.
- https://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.useshellexecute(v=vs.110).aspx
- ProcessStartInfo.RedirectStandardOutput:
- true if output should be written to Process.StandardOutput; otherwise, false. The default is false.
- https://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.redirectstandardoutput(v=vs.110).aspx
- ProcessStartInfo.RedirectStandardError:
- true if error output should be written to Process.StandardError; otherwise, false. The default is false.
- https://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.redirectstandarderror(v=vs.110).aspx
- CreateNoWindow:
- true if the process should be started without creating a new window to contain it; otherwise, false. The default is false.
- https://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.createnowindow(v=vs.110).aspx
- CreateNoWindow works in conjunction with UseShellExecute as follows:
To run the process without any window:
ProcessStartInfo info = new ProcessStartInfo(fileName, arg); info.CreateNoWindow = true; info.UseShellExecute = false; Process processChild = Process.Start(info);To run the child process in it’s own window (new console)
ProcessStartInfo info = new ProcessStartInfo(fileName, arg); info.UseShellExecute = true; // which is the default value. Process processChild = Process.Start(info); // separate windowTo run the child process in the parent’s console window –-> This is what we are using
ProcessStartInfo info = new ProcessStartInfo(fileName, arg); info.UseShellExecute = false; // causes consoles to share window Process processChild = Process.Start(info);
- WindowStyle :
- is a recommendation passed to the process
- One of the enumeration values that indicates whether the process is started in a window that is maximized, minimized, normal (neither maximized nor minimized), or not visible. The default is Normal.
Git branch cleanups
- git remote prune origin –dry-run
- (Local) git branch -d <localbranch>
References:
