Ad

Saturday, March 31, 2018

Command Line Essentials Cheat Sheet

  • echo $PATH
  • changing the $PATH to optimize for basic unix commands can help save time. Very important for server maintenance.
  • // print current working directory
    $pwd
  • grep -n showing line number
  • less slash search
    wc word count
  • If need to approximate find or remember a command:
    apropos keyword
     apropos copy
  • PHP Command line: error log (have to turn the error log on first)
  • >tail -100f error.log
  • Generally heroku and phonegap may require commandline tools, in heroku it is known as the heroku command line toolbelt
  • Show and hide hidden files on mac
 

MAKE HIDDEN FILES SHOW/HIDE IN MAC FINDER

Just follow the steps below and your finder will show hidden files:-

Start your “Terminal”
type “defaults write com.apple.finder AppleShowAllFiles TRUE” and enter (without the quote)
type “killall Finder” and enter (without quote)
Done. You have just enable your show hidden files in Finder
To disable the show hidden files in Finder, follow the steps below:-

Start your “Terminal”
type “defaults write com.apple.finder AppleShowAllFiles FALSE” and enter (without the quote)
type “killall Finder” and enter (without quote)
Done. You have just disable your show hidden files in Finder

Testing Toolbox

Selenium
Record clicks
insert blank line
s put an arrow on the line, play will handle it

JavaScript toolbox


  • Learn JavaScript for coding bootcamp graduates

  • Machine learning with JavaScript deep learn js with Tensorflow

  • Studying and teaching materials for JavaScript MVC
  • http://www.teaching-materials.org/jsmvc/
  • Learn the concept of JavaScript templates Knockout.js tutorial

Friday, March 30, 2018

Object Oriented Programming with Python

Define a Stack class, just the skeleton, illustrates how classes are implemented in Python. Lower case self, is the first argument to methods.

Class Stack:
def __init__(self):
pass
def is_empty(self):
pass
def push(self, value):
pass
def pop(self):
pass

Python toolbox


  • Scrapy for web scraping
  • Geopy for geographic data
  • SciPy for advanced science and mathematics formula
  • Numpy for linear algebra and matrix manipulation
  • Tensorflow for machine learning with CPU, GPU and in the cloud, with TPU
  • spaCy natural language processing NLP with python
  • control+cmd+g expand current word selection to all matching words.

Thursday, March 29, 2018

Handling Date and Time in Pandas

First of all need to convert DataFrame Series to datetime object. You can start by importing the datetime module.

import datetime

You can check the column data type of any Pandas DF columns

df['col_name'].dtype

It likely returns 'O' for object

In order to convert a column of type object to datetime use the pd.to_datetime function. You must also pass in a format specifying the year, month, day

df['col_name'] = pd.to_datetime(df['col_name'], format="%m/%d/%y")

This results in a datetime64 type column.

Formatting cheatsheet can be found here:
http://strftime.org/
For more advanced datetime formatting, also refer to the official document above

Pandas can sometimes infer or guess the datetime without receiving format instructions
use this flag below

df['my_col'] = pd.to_datetime(df['my_col'], infer_datetime_format=True)

Once converted to datetime, we can access year month day  data:
df['dt_parsed'].dt.day


Python Pandas Data Science Manipulation Pandas Cheat Sheet

pandas.Series = (
 {key : value}
)

pandas.DataFrame  = (
{key : value},
{key : value},
{key : value},
index=[]
)

Add new column
pandas.DataFrame['col_name'] = value

     value can be a series of vectors or dictionary
     pd.Series({0:'first', 1:'second'})



placeholder strategy
df['feedback'] = ['+', None, '-']

turn index into auto incrementing default numbers
guaranteed unique
df.reset_index()


join dataframes

Pandas data types http://pandas.pydata.org/pandas-docs/stable/basics.html#dtypes

Sunday, March 25, 2018

Fill forward fill backward pandas python financial analysis

Financial analysis data preprocessing using Python Pandas. Fill Forward and Fill Backward.

ffill(inplace=False)

bfill(inplace=False)



Android Basics 101

Android Basics and Keywords

Android Activity: a layout template MainActivity, DetailActivity, SettingsActivity

How to write and track a todo task in Android Studio:

To note a todo task and have it show up in the TODO Android Tools later use this format: 
// TODO : Implement this method
Note that the task is specified with a capitalized "todo" in the comment section, and the name of the task follows immediately after.

Styling in style.xml and Choose a Theme

Specify styling, layout and themes in style.xml. 

Example:
<item name="colorPrimary">#hex_color</item>
<item name="colorButtonNormal">#hex_color</item>

Android Code Patterns

findViewById(resource_id)

Tricks Android Studio can do

  • Search through all of Android Studio Cmd+Shift+A. 
  • Automatically reformat code with one click
  • Take a screenshot from device Android Monitor > Capture mode
  • Record screen for up to  3 minutes and output as a video of the hardware device Android Monitor > Screen Record
  • Presentation Mode: hide the toolbars
  • Android clip board tracks the copy-paste done in the past. 

Android Best Practice

  • Check for Updates frequently to get the latest Android Studio
  • Read Release Notes 
    • https://developer.android.com/studio/releases/
    • https://en.wikipedia.org/wiki/Release_notes
  • Mobile device has limited memories
  • Large images buildup can cause mobile applications to slow and even crash
  • Use LinearLayout Orientation Horizontal to position buttons or views in a row or in a column
  • Name conventions prefix state variables with small "m" as in "mContext".  "m" stands for member variable.
  • context is a special parameter that helps methods access state information in an instance of a class. Can be passed into TextView as a parameter. Helps access application resources and environment. 
  • https://developer.android.com/distribute/best-practices/
  • https://developers.google.com/training/android/
  • https://developers.google.com/training/courses/android-fundamentals
90% of your time as an Android developer will be spent in Android studio.

Java Coding for Android

The convention is constants are written in all CAPS. 

Declare a Java Class

https://docs.oracle.com/javase/tutorial/java/concepts/class.html

Declare a Java Method quickly https://docs.oracle.com/javase/tutorial/java/javaOO/arguments.html (Oracle)

Java keywords. Reserved symbols that has existing meaning / purpose in Java.  programming https://docs.oracle.com/javase/tutorial/java/nutsandbolts/_keywords.html

Basic Android Development Cheat sheet on Medium
https://medium.com/@ryanstewartalex/basic-android-development-cheat-sheet-58d501dabb3f

Common Android Views Cheat Sheet by Udacity
http://labs.udacity.com/images/Common-Android-Views-Cheat-Sheet.pdf

OOP in Java Android

Use @overide to declare overriding of parent methods. 
Use extend to inherit parent methods. 

Java strings are immutable, means they cannot be edited despite that indexing is allowed. The entire variable has to be reassigned.  StringBuilder is mutable and is used for that purpose.

Pro Tip turn on auto import to automatically import modules.

Variable scope matters. Each function variable will get destroyed if the function returned.

Android studio has left, right (aka side panes) and bottom panes - lots of easy access menu.

Android Getter and Setter Methods

Getter methods are traditionally prefixed with get. E.g. getText
Setter methods are traditional prefixed with set. E.g. setText()

Android Casting and Inheritance

casting inheritance
https://plus.google.com/+KatherineKuanPlus/posts/GyrcP6fEo2v
https://docs.oracle.com/javase/tutorial/java/IandI/subclasses.html
https://stackoverflow.com/questions/44902651/no-need-to-cast-the-result-of-findviewbyid

Android Log

Example: Log.i("name_of_class_where_error_is_from.java","Your Message Here");
https://developer.android.com/studio/command-line/logcat


Commonly used Android tools

Android Monitor - Android debugging system, output of log, breaks and code step through
Gradle - Gradle tasks that help project build and run correctly. Gradle is the system that is responsible ultimately for building and running Android projects.
Project -
Terminal - provides command line interface
TODO - track todo list in the source file.
Capture - tracks bugs found in connected devices
Event log - display messages related to events and activities like a notification system
Favorites - a favorite list for files
Gradle Console - display information about the Gradle system, display errors, warnings or success message
Message - Display outputs from the Gradle Build System. Display if any problem compiling the application
Structure - Shows high level view of the source file, classes, variables and methods. It's "hyperlinked" : can click and jump to source file.

A temporary message such as a notification is called a Toast in Android.

Android can use intent to activate functionality of another app: for example in web browser, you click on share, wanting to send an article using the Gmail app instead.

Common intents in Android

https://developer.android.com/guide/components/intents-common
https://developer.android.com/guide/components/intents-filters#Resolution
public void capturePhoto(String targetFilename) {
    Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    intent.putExtra(MediaStore.EXTRA_OUTPUT,
            Uri.withAppendedPath(mLocationForPhotos, targetFilename));
    if (intent.resolveActivity(getPackageManager()) != null) {
        startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
    }
}
See the code above. if intent.resolveActivity(getPackageManager()) == null is the exception handling.

Android Intent Exception Handling

Run an intent only if it is found in the system and can be resolved.

Android Intent is useful when you want to launch existing Android functionalities such as Camera, Send an Email, and more. 

Special formatted data can be written in (Uniform Resource Identifier) URI - structure data in a specific way so that it can be processed by the receiving end. For example tel:415-xxx-xxxx where x is some number between 0-9 is considered a phone number on iPhones. When user tap the URI, a phone call will be made. Similarly on android, tapping an URI linked email address will open up the Gmail app.

Commonly Used Android Methods and Attributes

findViewById returns a View
TextView.setText: change text of a TextView
ImageView.setImageResource: change image url source
onClick attribute: an XML field and an event listener on the android view

Common Android Errors

Android incompatible type error: usually fix by casting e.g. trying to store an int returning function result into a string variable

Android Developer Tools

Android SDK Search Chrome Extension: use this chrome extension to view Android source code
https://plus.google.com/+AndroidDevelopers/posts/1EyxkFnPjmD

You can also click on the view source link by each class name to access the source code in git.
https://android.googlesource.com/platform/frameworks/base/+/refs/heads/master/core/java/android/widget/TextView.java
https://android.googlesource.com/platform/frameworks/base/+/refs/heads/master/core/java/android/widget/ImageView.java

Udacity simplified TextView and ImageView class (teaching aid)
https://gist.github.com/udacityandroid/bd550cc8fd37190d85a6
https://gist.github.com/udacityandroid/47592c621d32450d7dbc


Read more here http://www.techotopia.com/index.php/A_Tour_of_the_Android_Studio_User_Interface

Saturday, March 24, 2018

Intuition for Principal Components Analysis - Georgia Tech and Udacity - Machine Learning

leetcode 83 remove duplicates from sorted linked list




tf–idf time frequency inverse document frequency Natural Language Processing Python Sklearn

TFIDF models how important keywords are within a document and also in the context of a collection of documents and texts known as a corpus. TFIDF is the key algorithm used in information retrieval. The importance factor is proportional to the frequency of the keyword appearance in the document, can be normalized by the length of the document, and then the inverse part: it is offset by how frequently the word appears in other documents in the corpus. This way naturally more frequently appearing words can be discounted such as economics, economy, etc in the economist magazine collection.

Note this may be frequently appearing words that are not stop words like the, a, and, however. Best practice of preprocessing text data may already include removing stop words and changing texts into lower case using .lower(). 

A bit of a tangent: some times, lower case removes meaning of nuances of words such as Anthropology. Capitalized Anthropology could mean the subject at college, the brand, but lower case anthropology could just mean: human societies, cultures and development. Even if we use stemming, we may never know that the author of the social media post actually is referring to a proper noun and a brand Anthropology, or Fossil. 

tf-idf is a popular term weighting scheme. Think Google Search, SEO, ranking of search results, NYTimes article text summarization. One can definitely develop fancier algorithms on top of this elegant and powerful concept.

Term frequency (TF): It's intuitive. The more often a word appears in a document, the more likely that's a part of its main topic. Caveat 1: keyword spamming. Caveat 2: what if document_1 is much longer than document_2? You can normalize the term frequency by document length. 

Inverse document frequency (IDF): Stop words like the, and, a appear very frequently in English texts, so regardless of whether they are useful in determining the actual meaning of the document, they will score high in Term Frequency. Remember our Economist Magazine? The word "Economist" may appear in the margin of every page spread. It's not helpful to help us distinguish article_1 and article_2. We may have to discount it. 

How to calculate TF-IDF by hand?

See this wikipedia screenshot https://en.wikipedia.org/wiki/Tf%E2%80%93idf

Note the very interesting case where "the" appears in every document so inverse document frequency  = log (number of docs in the corpus divide by number of docs containing the word "the") = log(2/2)  = log(1) which is 0! So this stop word does not matter at all in our text analysis task. 

Natural Language Processing (NLP) in general and with sklearn:
Tokenization: breaking sentences into words, and often take the count of the words. sklearn.CountVectorizer()

Here's a nice tutorial series on how to tokenize, stem, remove stop words using nltk library, a popular python natural language processing library.
https://www2.cs.duke.edu/courses/spring14/compsci290/assignments/lab02.html

It also shows how to marry tokenization and stemming with sklearn tf-idf  frequency inverse document frequency  TfidfVectorizer

Iterate through dictionary key and value The Pythonic Way

In Python 2.x you can iterate through a dictionary using:
for key, value in my_dict.iteritems():

Python 3: there is no more iteritems, items() achieve the same result which is returning an iterator - see below for more explanations.

for key, value in my_dict.items():
     print(key,value)

For Python 2.x

  • dict.items(): Return a copy of the dictionary’s list of (key, value) pairs.
  • dict.iteritems(): Return an iterator over the dictionary’s (key, value) pairs.
Some may say iterating through a large dictionary is not a good idea in the first place and is not Pythonic. 
See Python Anti-Pattern https://docs.quantifiedcode.com/python-anti-patterns/performance/not_using_iteritems_to_iterate_large_dict.html

Python doc has some suggestion to help handling legacy code in Python 2.x
https://www.python.org/dev/peps/pep-0469/

try:
    dict.iteritems
except AttributeError:
    # Python 3
    def itervalues(d):
        return iter(d.values())
    def iteritems(d):
        return iter(d.items())
else:
    # Python 2
    def itervalues(d):
        return d.itervalues()
    def iteritems(d):
        return d.iteritems()

Best intuition on Max Pooling - Image Processing technique

If you read about Image Processing and machine learning  using Convolutional Neural Network, you probably have heard of Max Pooling. This blog has amazing visualizations, see link below. This is their visualization for Max Pooling.  Really helpful and easy to understand.

Sources: http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

Android Basics: why isn't String type lower case?

Sample code String message = "my message here"; why isn't the string type declaration lower case like int? Because String is not a primitive data type. Some primitive data types include float, long, int. See android developer official documentation here https://developer.android.com/reference/java/lang/Number.html

Analyzing Anonymous Features

Can try decode features. It is important to guess the feature types to be on the right track of feature processing and also selecting the right model. Anonymous data processing is a true, advanced way to test data modeling skills of any data scientists. Intuition, domain knowledge may not apply well in this situation. It's a true fun challenge.

Data Preprocessing using Pandas - Exploratory Data Analysis (EDA) cheatsheet

Helpful pandas functions for data quality:

  • DataFrame.dtypes()
  • DataFrame.info()
  • DataFrame.describe()
  • x.value_counts()
  • x.isnull()
  • x.mean()
  • x.var()

Wednesday, March 21, 2018

WordPress 4.8 Lastest Release

The newer WordPress latest release -WordPress Evans- gets a new default theme, and is named after a Jazz musician William John "Bill Evans" -

Easy to understand explanation of Heap and Priorities Queues

A great explanation from youtube : explanation of Heap and Priorities Queues for beginners, coders, learning to code, bootcamp graduates. Semi ordered. Parents, direct children and grandchildren follow this order.

Difference between iloc and loc in Pandas

ix is deprecated, use of iloc and loc are encouraged.

.loc for label based indexing or
.iloc for positional indexing

Useful for querying row data
df.head(5)
df.loc[0:5]
Also powerful for column based querying
df_pro_desc.loc[0:2,["col_1", "col_2"]]

Teach Your Kids to Code!

Hello Ruby: Adventures in Coding Linda Liukas debut on Kickstarter. Beginner friendly, story telling




Code It! Create It!: Ideas & Inspiration for Coding (Girls Who Code) Hardcover – by local meetup nonprofit that promotes girls who code.





Coding iPhone Apps for Kids: A Playful Introduction to Swift - Gloria Winquist. Published by Starch Press Comprehensive book on iPhone app programming, kids friendly.
 

Teach Your Kids to Code: A Parent-Friendly Book by Bryson Payne. Also published by Starch Press.



Scratch Programming Playground: Learn to Program by Al Sweigart. Also published by starch press



Tuesday, March 20, 2018

What is Stoi ?

Read more tips for coding interviews for bootcamp graduates on our blog.

Stoi reads String to Integer. it's a function that converts string to integer e.g. "25" --> 25 "01" --> 1. You may be asked to implement this in an interview.

Sunday, March 18, 2018

Elon Musk's Artificial Intelligence Research Dream Team

Did you know that Elon Musk is involved with OpenAI? In fact, OpenAI is managed by Silicon Valley titans such as Elon Musk, Sam Altman and Jessica Livingston of Y Combinator, Reid Hoffman founder of LinkedIn, Peter Thiel and Greg Brockman previously CTO of Stripe! OpenAI claims to do AI research and publish at key AI industry conferences. It also boasts sponsors such as infosys, microsoft, aws, asana, atlassin, buffer, cloudflare, github, cisco, nvidia, ... many of which are YCombinator prodigies. Here's its press page explain what are its long term and short term goals. https://openai.com/press/

Monday, March 12, 2018

Clean Check Is Palindrome Implementation with Python

def is_palin(string, i, j): 
 while j > i: 
   if string[i] != string[j]: 
     return False
       i += 1
       j -= 1

Sunday, March 11, 2018

Creator of GAN teaches Udacity Deep Learning course

We notice this crucial info for our Deep Learning and Machine Learning students. Udacity Deep Learning Nanodegree explores creator and maintainer of GAN aka Generative Adversarial Network  - Ian Goodfellow - to teach its Deep Learning nanodegree. Ian was an experienced teaching assistant  at Stanford University. His also the author of Deep Learning published by MIT Press.


Entropy, Cross Entropy, Information Gain Explained by Stanford University Lecturer





entropy: how spread out is the distribution. many bits to transmit info. randomly uniform  distribution (see our 1 minute post on randomly uniform distribution) have a lot of entropy, uniform distribution very little. e.g. if i tell you the price starts with 0 increase by 5% every day. that's very few summary stats telling a big distribution. Versus, i can't describe random.

Understand the Softmax Function in Minutes

Reposted from Uniqtech's Medium publication with permission. This is retrieved on May 14 2019. Uniqtech may have a newer version. Unde...