Women in Data Science Conference (WiDS) summary, transcripts, notes from my personal experience

WiDS started small but is now a global movement with many regional events and branches. It is 5 years old in 2020. This year it is hosted at Stanford University.

Volunteer opportunities with WiDS: ambassadors, region events and branches, 500+ ambassadors world wide

Understand the history and evolution of Tensorflow by revisiting Tensorflow 1.0 Part 1

Tensorflow 2.0 has been beta since last year, and it is a completely different universe as its predecessor Tensorflow 1.0 but even in 2020 it is important to understand the history and evolution of TF library to understand how did it get from here, and why did it choose Keras as a high level API. It is important to understand what is a compute graph as it is a super useful concept in Deep Learning and that you can still visualize and inspect it in TensorBoard.

Let's go back in time and talk about Tensorflow 1.0 its data flow graph and everything executed in a Session object, C or C++ backend and how it handled parallel computing. Offered both Python and C++ API. Though it had a big learning curve at the time of its release, it was production ready and powerful, and had already been used internally at google before being outsourced. It supported CPU GPU and distributed processing in clusters. Its focus is on deep learning neural networks versus Scikit Learn focuses on traditional machine learning algorithm.

What is a data flow graph? It is a very important computer science concept. The node represents math operations, and the edges are multi dimensional arrays called tensors, which flow among the data graph hence the name Tensorflow! See the history is important. Using the graph we can easily visualize the neural networks. Numpy and Scitkit learn would not give that result.

The frustrating part is that graph needs to be built first before running it in a Session. This is where the learning curve got a bit hard and that it was hard to prototype and iterate, and requires a bit of math architect skills than just engineering and coding.

A quick note on tensor, which is also a concept in math and relativity. In this case, it just means a more complex multi-dimensional array of numbers, usually more than 2D (matrix), and has auto gradient compute compatibility and also capability to move to CPU or GPU and parallelize vector compute if possible. Technically even a vector (1D) or a number (0D) is a tensor.

In deep learning, usually we have to convert data such as texts or images into integers, and we usually represent them using tensors. Each image for example is a 3 dimensional tensor of red green and blue. Each dimensional has a matrix corresponding to the width and height of the image, with each element representing the pixel brightness at each w, h coordinate. This is called a feature matrix, aka a feature tensor. Though no one calls it a tensor in this case.

One cumbersome pattern in Tensorflow 1.0 was the need to define a placeholder tf.placeholder() and with type specified before filling it or initializing it with actual numbers. It has the benefit of contiguous memory but it can take time to get used especially when TF is trying to court dynamic type python users. The benefit is also to be able to construct the graph without knowing or filling in specific numeric values. One minus is the inability to test and prototype and iterate.

tf.Variable() allows initializing and filling in data that can be later changed. Each node is an unit of computation. Each edge is either an input or output of an operation.

tensorflow 1.0 like tensorflow 2.0 has a pythonic front-end, a pythonic API and can be deployed on many containers and devices such as CPU, GPU Android and other mobiles OS such as iOS, javascript (tensorflow 1.x+)  in the browser. It has always been quite production ready. Hence was popular before Pytorch 1.0 came along.

Tensorflow and Pytorch both focuses on deep learning and are optimized for deep learning.

Additional feature - Auto differentiation is important for gradient based deep learning algorithms. Additional feature - Optimizer for fine tuning weights efficiently.

SpaCy for Natural Language Processing (NLP)

Documents are tokenized to sentences, then to words. Additional or readily available features can be made from these documents to work a task. One SpaCy task could be identify whether a tweet is positive or negative and among its texts, is there a specific product that is mentioned.

Installation

Install SpaCy with Python like any other python package using pip. The easiest way to install and configure is to use Tensorflow Colab. 

pip install and then import SpaCy. It works on Tensorflow Colab too. Perhaps the fastest way to get started.

Step 1. Need to import a language model before proceeding. SpaCy supports many language models.

Step 2. Load the English model:

spacy.load('en')

spacy.blank('en')

Supports other models too, include en_core_web_sm

Update the model (optional)

nlp.update

Step 3. Init the model, wrap it in an nlp object

doc = nlp(u"document sentence here")

print out items from the spacy nlp model

- print out tokens
# Iterate over tokens in a Doc
for token in doc:
    print(token.text)

- print out entities

for ent in doc.ents:

print(ent.text, ent.start_char, ent.end_char, ent.label_)

this will print out the entity as well as the beginning and ending index and its label

Can also query doc using slicing example [1:4]
Can access .text attribute fo the token

Additional functions

Pipeline

print(nlp.pipeline)

Disable pipeline for custom training

nlp.disable_pTraining src 13

source -13 : https://spacy.io/usage/trainingipes
Why disable the pipeline? Say if you are using SpaCy for just one task such as NER, you can disable the pipeline to avoid some of the tasks.
Check the list of pipeline labels nlp.pipe_names()

Save the trained model

nlp.to_disk

NER
Bloom embedding, a type of optimized word embedding
1D CNN 1D convolutional neural network


NLTK
Not a part of Spacy is the entry level tool kit to NLP. It can do basic part-of-speech tagging. But does not have advanced functionality like spacy

Spacy Deep Learning

Training src 13

source -13 : https://spacy.io/usage/training

SpaCy word embedding

To print out the word embedding 

print out vector, access word embedding use .vector method

SpaCy for Biomedical Research

scispaCy, a python package that provides SpaCy models for biomedical, clinical texts and scientific literature. Pre-processing. 

Source: https://allenai.github.io/scispacy/

Why Natural Language Processing is hard?

Limitation of pre-trained models

" A model trained on Wikipedia, where sentences in the first person are extremely rare, will likely perform badly on Twitter. Similarly, a model trained on romantic novels will likely perform badly on legal text." - Spacy documentation

OpenCV cheat sheet

• import cv2
• cv2.imread()
• cv2.resize()
• .tranpose() on arrays
• .reshape() on arrays

Google Cloud AutoML

• Functionality provided by AutoML: Single Label Classification - 1. Predict the ONE correct label that you want to assign to a document 2. Multi-label Classification Predict ALL the correct labels that you want to assigned to a document. 3. Entity Extraction Identify entities within your text items. 4. Sentiment Analysis understand the overall sentiment expressed in block of text. (Source direct quote AutoML documentation). 

My experience with TripleByte technical interview and quiz

I read a few really good posts on TripleByte experience. They were helpful so I am also posting my two cents here.

First of all, TripleByte is legit. It went through Y Combinator and it is being actively promoted by YC.

Amazing selection of quizzes:
I am so happy that they have full stack, data science as well as Machine Learning quizzes as of Feb 2020! The Data Science and Machine Learning Quizzes both have a NEW sign.

It is about 2 minutes per question.

I really like the FastTrack feature. It is a quick validation. It is encouraging and it quickly moves candidates to the next step  : actually doing or practicing technical interviews. Honestlly this part is not avoidable.

I haven't figured out a way to take other quizzes when passing one with FastTrack.

It is not very hard for me to get FastTrack or well but if I can get exceptionally well, then it is rarer and more meaningful, and there may even be an opportunity to be matched with top companies and opportunities. I don't think the Exceptionally Well is exactly trivial to obtain. TripleByte visualizes your skill set with sub categories that either has a scale of 1-5 rating or a radar map with similar scale. But one does not want to score a 3 in any of the sub categories - visually it makes the radar map looks weak.

With a little a bit of review and brief study, the quizzes should be easily passable. If you don't pass the quiz, may be it is time to learn more and get more experience, because it is not that hard to pass it.

From most of what I gathered online in forums, the technical interview portion is difficult. There is quite a bit of requirement in coding exercises and setting up the coding environment in the console. Because I come from a non-traditional background, I don't know C++ ... yet. I plan to learn it. Some of the exercises, quizzes and interview questions can be in C++. And that's a problem for me. The quiz C++ is easy to figure out even if you don't know the language. But the coding exercise in C++ cannot be figured out without prior knowledge.

Apparently you will be sent an interview guide if you do schedule a technical interview.

One trick to do well in technical interview is to have practiced the problem, then you will know the caveat, and won't stress to understand the problem (comprehension), and potentially know roughly what the optimal solution look like.

During the interview, it'd be good to think of a similar problem that you resolved and recall how you resolved it. Being able to discuss the problem in a real world setting is always helpful for finding optimal solution and also showcase your understanding of the technical problem.

How does TripleByte compare to HackerRank and Leetcode

TripleByte is more developer-friendly and better for candidates than HackerRank and Leetcode. Because first of all, it tests knowledge more than trivia. As long as you understand the problem, you likely can resolve the question fast, within 2 minutes (the requirement). It focuses one or two missing line, or the final returned result. This means you won't have to spend 45 minutes to conjure each solution. I like that a lot. I can demonstrate I understand the problem and its edge cases without having to get very detail right. 

Leetcode is more detailed, and there is a lot of competition for time performance, even a good solution may not be enough. HackerRank has a nice trajectory to level up, and is interesting, but like Leetcode it also requires the candidate to write a lot of code every time. Though eventually, you should probably still use HackerRank or Leetcode to prepare for the screenshared interview - first round. 

HackerRank supports a few choices of languages. TripleByte lets you choose category of your quiz but there no explicit language choice. Leetcode supports many languages. 

Unit Testing

Unit testing usually focuses on a single function or a small chunk of code.

Writing and fixing unit test code early can have the effect of managing bugs, errors and failing logic before they get out of hand.

Writing dedicated unit testing code is better than simply testing code interactively, which is a manual not repeatable process.

Test Driven Development (TDD) process likely will require developers to write the testing code first, expect it to fail, and then fixing the failing unit test before progressing.

Test automation is important because the number of unit tests can grow fast with new added features and functionalities.

It is different from regression testing and system integration testing, which both test the entire system more extensive including changed and unchanged parts.

It is also possible to be asked unit testing and testing questions in general in interviews.

Py test convention is to name the testing file similarly with the code file with a test suffix.