Facebook Pytorch Deep Learning Artificial Intelligence Basics

Pytorch is a deep learning framework and library promoted by Facebook. Tensorflow is promoted by Google. Both are gaining momentum and market share. Both are open source, Tensorflow is released in 2015, and Pytorch a year later in 2016. TF may be the winner based Github statistics but Pytorch is gaining popularity and traction.

Remember to always zero out gradient when done training, or in an epoch, or certain number of iterations, else the gradient will accumulate over time.

Pytorch convert torch tensor to numpy ndarray and numpy array to tensor

Google AMP for Fast Performant Mobile Website - Front-End Development

AMP aims to amp up mobile website performance on IoT, web both mobile and desktop. "Content is always King" "User Experience is always queen." AMP has an open sourced code base, in collaboration with big shot startups and tech companies. It writes like normal HTML, with some custom tags such as <amp-img> versus <img>.  Every web application enabled with AMP contains an Amp js file, which controls the entire load chain, result in a few optimized behaviors: top content is loaded first so users can start reading and viewing right away, layout of the page is pre-calculated based on device and viewport height width (placeholder images will be placed at the perfect location when not loaded)

Related concept Response, Animation, Idle, Load (RAIL) model. Resource https://youtu.be/lBTCB7yLs8Y?list=WL

Deep Learning Papers that you must know!

Word embedding: Mikolov et al., 2013b; Pennington et al., 2014 word level embedding. One embedding per language versus LASER one embedding for all languages.
Sentence embedding: Conneau et al., 2017; Peters et al., 2018; Devlin et al., 2018 sentence level embedding. One embedding per language versus LASER one embedding for all languages.

NLP
Word2vec

Paper on data augmentation. Data augmentation scaling, rotations, mirroring, and/or cropping. The Effectiveness of Data Augmentation in Image Classification using Deep Learning Jason Wang Stanford University Luis Perez Google
http://cs231n.stanford.edu/reports/2017/pdfs/300.pdf

AlexNet (wikipedia) : "AlexNet is the name of a convolutional neural network, designed by Alex Krizhevsky,[1] and published with Ilya Sutskever and Krizhevsky's PhD advisor Geoffrey Hinton,[2][3] who was originally resistant to the idea of his student.[1][4]

AlexNet competed in the ImageNet Large Scale Visual Recognition Challenge[5] on September 30, 2012. The network achieved a top-5 error of 15.3%, more than 10.8 percentage points lower than that of the runner up. The original paper's primary result was that the depth of the model was essential for its high performance, which was computationally expensive, but made feasible due to the utilization of graphics processing units (GPUs) during training.[5]" AlexNet is similar to modern CNN architecture made significant improvement in ImageNet competition performance!

Plagiarism detection: Clough, P. and Stevenson, M. Developing A Corpus of Plagiarised Short Answers, Language Resources and Evaluation: Special Issue on Plagiarism and Authorship Analysis, In Press. 

Import Sklearn libraries and functions

from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from lightgbm import LGBMClassifier
from category_encoders import OneHotEncoder
from sklearn.model_selection import cross_val_predict
from warnings import filterwarnings
from sklearn.impute import SimpleImputer

Differential Privacy and Federated Learning 101

A list of glossary, vocabulary for differential privacy, secure AI, and federated learning on sensitive dataset. All concepts are theoretical, for discussion purpose only, are NOT intended for production nor any professional usage, and should NOT be used. This is a very new experimental concept in AI.

Before you read our article please read our disclaimer page. It is very important to note that this article and all articles, content on our website and affiliated websites are for discussion / entertainment purpose only. They should NOT be considered professional advice. Content on this site, our affiliated sites and social media are NOT intended for commercial purpose; NOT for production purpose; NOT for professional usage.

What is a scenario that differential privacy is useful? When a researcher wants to analyze a sensitive dataset, such as a dataset containing patient data, and or when a research wants to make a model that learns sensitive features, and or make a sensitive prediction: i.e. if a patient has a health issue, such as HIV.

Differential Privacy

"mathematical definition of privacy. In the simplest setting, consider an algorithm that analyzes a dataset and computes statistics about it (such as the data's mean, variance, median, mode, etc.). Such an algorithm is said to be differentially private if by looking at the output, one cannot tell whether any individual's data was included in the original dataset or not. In other words, the guarantee of a differentially private algorithm is that its behavior hardly changes when a single individual joins or leaves the dataset " - Harvard Differential Privacy Group

Query:
See the above quote
Mean variance medium mode
Advanced: machine learning and deep learning model.

Differentially private tools

Sensitive database

Anonymization: the old way, removing sensitivity personally identifiable information, has shown to sometimes fail. No guarantee.

Example where data thought to be anonymous, fails to protect privacy:

 Latanya Sweeney showed gender, date-of-birth, zipcode can identify many Americans. This is also the governor medical record example. This is known linkage attack.

Linkage Attack:
See above

"too many innocuous (even completely random) queries about a database inherently violates the privacy of its individual contributors. ... tradeoff between statistical utility and privacy." - Harvard Privacy Group

owner: the initial virtual owner is me
pointer API for send() and get() data among virtualworkers.

Pysyft https://github.com/OpenMined/PySyft

Deep Learning using pysyft

Tech Digest Summer-Fall 2019

Google Cloud

Kubernetes Clusters "What's a cluster? It's a set of master components that control the system as a whole and a set of nodes that run containers. In Kubernetes, a node represents a computing instance. In Google Cloud, nodes are virtual machines running in Compute Engine. " - Google Cloud

Service group pods together. Each pod usually contains one container.

Public IP: allows outside visitors to access Google Cloud resources.

Deep Learning 

- Input times weight add a bias activate. 

- Deep learning at home: GTX 1080 Ti
Deep Learning Neural Networks
- Inpspired by neural networks
- These are artificial neural networks
- Connected nodes, with edges
- Model sued to estimate and or approximate functions
- Need large number of inputs

Graph Convolutional Network (GCN) : 

- Adjacency matrix is the input to GCN

- Geometric deep  learning versus 2D euclidean learning

- RNN is a chained graph, replace each unit with a recurrent unit

Facebook NLP multiple languages
https://ai.facebook.com/tools/fasttext


What function turns products into sums? Choices: Sin, Cos, Log, Exp. Answer log. Log will turn products into sum. [Deep Learning math tricks]

Secured & Private AI

Private machine learning on data you are not supposed to see. 

Expression

How to resolve XYZ?

What would you do for XYZ?

How do you solve XYZ?

Best co-working place in the world

Best exhibitions at tech conferences

Technology

How to host Discourse Forum on Google Cloud? Bitnami implementation, monthly host, public ID static address with CNAME DNS configuration.

Server for Python backend : Tornado, there's also Tornado UI to aim with fast development

Host Keras machine learning deep learning models in browser: using keras js

Visualizing convolutional layers https://www.oreilly.com/ideas/visualizing-convolutional-neural-networks

Startup devtools: Marvel, VideoScribe, Youtube shorts, grapheneDB, graph adjacency matrix, 

"Discourse is an open source discussion platform with built-in moderation and governance systems that let discussion communities protect themselves from bad actors even without official moderators." - Google Cloud market place. Discourse smartly manages who can post and how often they can post. Advanced features are only available for reputable and situated community members. Bitnami provides some security measures.

FAQ for Machine Learning

Where to find open Jupyter Notebook directory?
Standard Scaler versus MinMaxScaler. Standard Scaler MinMaxScaler strength and weakness.
http://rajeshmahajan.com/standard-scaler-v-min-max-scaler-machine-learning/ MinMaxScaler takes into X.max and X.min so it is sensitive to outliers.

Tutorials

fast.ai
Pysyft tutorial, Andrew Trask in Slack
CS50
ICME
Qwiklab games

Data Science

Pandas

Pandas.Series.value_counts

Return a Series containing counts of unique values.

The resulting object will be in descending order so that the first element is the most frequently-occurring element. Excludes NA values by default

One Hot Encoding

# TODO: One-hot encode the 'features_log_minmax_transform' data using pandas.get_dummies()

features_final = pd.get_dummies(features_log_minmax_transform)

Use lambda function in Pandas

income = income_raw.apply(lambda x: 0 if x=='<=50K' else 1)

Neo4j

(Data Scientists) -[:LOVE]->(Neo4j)

nodes are in (), relationship is in [], colon : marks the label, -> is the direction

Publication pipeline

SageMaker Youtube shorts

Jobs

YC expo, startup expo jobs on Youtube live

Technical Interviews

O(n) is just m + m + m ... It is linear.

Growth Hacking

Youtube creators can now send messages to friends.