RichardOnData
Impostor Syndrome in Data Science
updated
In this video, I go through the different types of binary classification metrics. These include: accuracy, prevalence, confusion matrices, sensitivity (aka recall or true positive rate), specificity (aka true negative rate), precision (aka positive predictive value), F1 score, and the areas under the precision-recall curve and the receiver operating characteristic curve, that is: AUPRC and AUROC. We close with how to implement these using the scikit-learn package in Python, going through a Jupyter notebook.
Code can be found here: github.com/RichardOnData/YouTube/blob/main/Python%20Notebooks/classification_metrics.ipynb
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I talk about SHAP values and how these can be used for explainable AI and explaining how features contribute to a machine learning's predictions for each observation. These are great tools when your goal isn't (only) prediction, but is also inference - that is, understanding the most important features that influence a response or exactly one how feature, at different levels, influences the response. Note that this is different from causal inference, which is a separate and very complex topic altogether.
SHAP value documentation:
shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainable%20AI%20with%20Shapley%20values.html
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
NOTE: Sorry about the bad audio quality on this one. I switched microphones when I upgraded phones recently, and thought during testing that it would be a lot better than it was here. Looking into a REAL microphone upgrade here.
NOTE 2: I didn't talk about DALL-E on this one, which is another feature to GPT-4. The focus here was mostly on the type of use cases I assume most people are using ChatGPT for these days, as opposed to image generation.
In this video I discuss ChatGPT 4, and compare it to ChatGPT-3.5 which most people use by default. I make some comparisons, looking at the file upload feature as well as comparing the responses side by side, and then talk about who exactly should make the upgrade here.
In this video I'll break down some tips that I have to get data jobs. This is going to be broad and apply to all types of positions, whether those are data analyst, data science, or data engineering jobs!
To summarize:
1) Have good education in a field like statistics, computer science, math, engineering, business, or physics. The higher up you go, the better.
2) Get some experience. That might mean asking your network if they need help, or using Upwork or Fiverr and getting creative.
3) Tell good stories. Lay out: context, problem, data, solution, and impact.
4) Develop differentiating skills. I think there's value in data scientists getting more engineering skills and data engineers getting more analysis skills.
5) Slow down and think more about jobs you want. Don't apply everywhere; find jobs you're more interested in and give more effort to those applications and to honing the right skills for them.
AWS certifications: aws.amazon.com/certification
Azure fundamentals: learn.microsoft.com/en-us/credentials/certifications/azure-fundamentals
GCP certifications: cloud.google.com/learn/certification?hl=en
In this video I show some ways I've used ChatGPT to both learn, and to data science faster. ChatGPT can be an excellent tool if you're responsible with it. It can provide great ideas to help get through creative roadblocks, as well as to generate great coding examples that you can turn around and use to learn. You DON'T want to become reliant and dependent on it to write code for you!
My thoughts on the data job market in 2024. I looked at data scientist, data analyst, data engineer, and machine learning engineer jobs. In particular we talk about some broader trends in tech more recently, the recent tech layoffs, and what hiring and salaries are looking like for these positions.
Crunchbase: news.crunchbase.com/startups/tech-layoffs
InterviewQuery (December Update): interviewquery.com/p/december-data-science-job-market
McKinsey Tech Trends: mckinsey.com/capabilities/mckinsey-digital/our-insights/the-top-trends-in-tech
NOTE: The beginning of this video is somewhat tongue in cheek. Certain things, you just have to let yourself have fun with. Some of the articles and videos I reference make very different points, specifically regarding the rise of data engineering and constructing end-to-end machine learning pipelines. Those are valid points that I don't want to be dismissive of, but overall, the subject for a different video.
In this video I specifically want to focus on and give my opinions to a commonly raised concern, that of ChatGPT and generative AI displacing data science and data analyst jobs. Here's my thoughts on the broader question of will data science die in 5 years, specifically as a result of AI and LLMs taking said jobs.
In this video we're revisiting the R vs Python comparison in the year 2024. How do they stand in recent job reports and in indices like PyPL or the TIOBE index?
In this video I talk about every data science job I've had, how each job was dramatically different from the others, and how each one sort of led to the next.
In this video I'm going to provide a recommended 10 packages that you should know and focus on to get strong at Python programming, in the context of data science.
Recommended book "Python for Data Analysis": amzn.to/3cDXKcE
1. pandas
pandas.pydata.org/Pandas_Cheat_Sheet.pdf
2. numpy
images.datacamp.com/image/upload/v1676302459/Marketing/Blog/Numpy_Cheat_Sheet.pdf
3. matplotlib
matplotlib.org/cheatsheets/_images/cheatsheets-1.png
4. seaborn
images.datacamp.com/image/upload/v1676302629/Marketing/Blog/Seaborn_Cheat_Sheet.pdf
5. datetime
pythoncheatsheet.org/modules/datetime-module
6. statsmodels
statsmodels.org/dev/examples/index.html
7. scikit-learn
images.datacamp.com/image/upload/v1676302389/Marketing/Blog/Scikit-Learn_Cheat_Sheet.pdf
8. streamlit
docs.streamlit.io/library/cheatsheet
9. tensorflow
zerotomastery.io/cheatsheets/tensorflow-cheat-sheet
10. keras
images.datacamp.com/image/upload/v1660903348/Keras_Cheat_Sheet_gssmi8.pdf
In this video I cover survival analysis. Specifically what it is, and why it's useful when the time until an event is important and when you have "censored" data. I talk about what censored data is and provide definitions of the survival and hazard functions. This is illustrated visually by showing a Kaplan-Meier curve as well as the idea behind the logrank test. We close with modeling. There's a lot of different survival models, such as survival trees and survival forests. For parametric versions there's accelerated failure time models. I show an example of a Cox proportional hazards model and explain what exactly the (very important) proportional hazards assumption means.
Real world example: https://web.cortland.edu/matresearch/
Censored data example: blog.minitab.com/en/michelle-paret/the-difference-between-right-left-and-interval-censored-data
Kaplan-Meier Curve example: https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_survival/BS704_Survival_print.html
Logrank test example: https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Survival/BS704_Survival5.html
Cox model example: https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_survival/BS704_Survival6.html
Packages for survival analysis:
Python: github.com/CamDavidsonPilon/lifelines
R: github.com/therneau/survival
ChatGPT:
Bri Does AI: youtube.com/watch?v=MnDudvCyWpc
Ryan Scribner: youtube.com/watch?v=X9ksiScY7hM
Statistics:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
SQL:
"The Guru's Guide to Transact-SQL": amzn.to/4aMtmuf
R/Python:
"R for Data Science": amzn.to/3je8kK6
"Python for Data Analysis": amzn.to/3cDXKcE
Command Line:
"100 Command Line Tools": amzn.to/47iDhEV
Machine Learning:
"Hands-On Machine Learning": amzn.to/3eAh1N2
"The Hundred-Page Machine Learning Book": amzn.to/3563PfK
"Deep Learning": amzn.to/3p83vVW
Andrew Ng's course: coursera.org/specializations/machine-learning-introduction
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I walk you through how to set up your Python development environment. If you're a complete beginner, you'll probably be good with just Anaconda/JupyterLab/Jupyter Notebooks. If you're going to be a serious developer, you'll want to use Visual Studio Code and as a best practice set up virtual environments.
Anaconda: anaconda.com/download
Visual Studio Code: code.visualstudio.com/download
'Journey to Become a Google Cloud Machine Learning Engineer': amzn.to/3TjwmYT
Exam guide: cloud.google.com/learn/certification/guides/machine-learning-engineer
Github compilation: github.com/sathishvj/awesome-gcp-certifications/blob/master/professional-machine-learning-engineer.md
Medium articles:
towardsdatascience.com/a-comprehensive-study-guide-for-the-google-professional-machine-learning-engineer-certification-1e411db4d2cf
datacouch.medium.com/google-cloud-professional-machine-learning-engineer-certification-preparation-guide-2067478767ff
Hi everyone. It's been a while.
In this video, I discuss statistician Edward Tufte's famous 6 principles of graphical integrity. Credit to Radhika R for the excellent LinkedIn Pulse article displayed for the majority of this video.
Articles:
linkedin.com/pulse/edward-tuftes-six-principles-graphical-integrity-radhika-raghu
businessinsider.com/the-27-worst-charts-of-all-time-2013-6#welcome-to-fox-where-the-line-graphs-are-made-up-and-the-points-dont-matter-12
infovis-wiki.net/wiki/Data-Ink_Ratio
anilbas.github.io/teaching/hci/week13/Tufte.pdf
clauswilke.com/dataviz/no-3d.html
Books:
"The Visual Display of Quantitative Information": amzn.to/3A7bGtJ
#DataScience #BreakingIntoDataScience #StatisticsForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I discuss some of the reasons why it can be so difficult to get a data science job. I talk about skills and mismatches that you may have with those skills, the different titles that you might look for to optimize your chances, and getting through interviews and assessments.
Articles:
kdnuggets.com/2020/09/modern-data-science-skills.html
towardsdatascience.com/why-are-data-science-jobs-hard-to-get-when-theyre-the-most-in-demand-30eda0d1b5ec#:~:text=Data%20Science%20requires%20a%20discipline,jobs%20or%20computer%20science%20jobs.
towardsdatascience.com/5-reasons-why-its-difficult-to-get-a-job-in-data-science-8d2140766183
towardsdatascience.com/acing-the-data-science-take-home-assignment-530a93164e52
My videos referenced:
"How to Get a Data Science Job in 2021": youtu.be/OgXNyZIka-Q
"A Study Pathway for Data Science in 2020": youtu.be/ySI5P37Xf_k
"Can You Become a Data Analyst/Scientist With No College Degree?": youtu.be/RSiKo7daQSU
Statistics Coursera courses:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
Books:
"An Introduction to Statistical Learning": amzn.to/3mzrwmf
"The Hundred-Page Machine Learning Book": amzn.to/3mBOSHK
"The Guru's Guide to Transact-SQL": amzn.to/34ven7L
"R for Data Science": amzn.to/3nOHieF
"Python for Data Analysis": amzn.to/37ySAOc
#DataScience #BreakingIntoDataScience #StatisticsForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I talk about a few months ago when I quit my data science job, reasons for it and parallels to the last time I did so. I have a new data science job today, but probably won't be in data science forever. I discuss why!
#DataScience #BreakingIntoDataScience #ProgrammingForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I talk about some good coding practices to be a strong programmer in data science. We talk all the time on this channel about the need to know at least one programming language like R or Python, but then once you're working with a language (any language), what are some of the best practices?
1) Define the purpose of the code
2) Minimize things that will (inevitably) need to be changed
3) Work from the top down, not to the right
4) Use descriptive variable and function names
5) Use functions rather than repeating code
6) But don't go overboard with chaining functions
7) Pay attention to datatypes
8) Provide adequate documentation and comments
9) Check for the latest and greatest packages for the task
10) Test as you go
#DataScience #BreakingIntoDataScience #ProgrammingForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
"How Much Statistics Do You REALLY Need for Data Science?": youtu.be/_tEek2jPn-4
"How to Get a Data Science Job in 2021": youtu.be/OgXNyZIka-Q
"A Study Pathway for Data Science in 2020": youtu.be/ySI5P37Xf_k
"Can You Become a Data Analyst/Scientist With No College Degree?": youtu.be/RSiKo7daQSU
In this video, I talk about how to break into data science in the year 2021, if you're a college student. This is a time when you have amazing opportunities around you, you just need to recognize them and take advantage of them!
Statistics Coursera courses:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
Books:
"An Introduction to Statistical Learning": amzn.to/3mzrwmf
"The Hundred-Page Machine Learning Book": amzn.to/3mBOSHK
"The Guru's Guide to Transact-SQL": amzn.to/34ven7L
"R for Data Science": amzn.to/3nOHieF
"Python for Data Analysis": amzn.to/37ySAOc
#DataScience #BreakingIntoDataScience #StatisticsForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I'm going to talk about all these "learn data science in 2 months / 3 months / 6 months" articles, courses, and videos that you see out there. I want to talk about the reality of learning data science and what it's more realistically going to look like. You want to think of it as a marathon, not a sprint. Learning is not a singular act. It's a mentality, not a process. And that's what will set you up best for the most fulfilling and lucrative experience with data science possible.
Coursera statistics courses:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
An Introduction to Statistical Learning: amzn.to/3k8SaRN
The Guru's Guide to Transact SQL: amzn.to/2IgvGko
R for data science: amzn.to/3je8kK6
Python data science book: amzn.to/3cDXKcE
Linear algebra: amzn.to/2HEj4U4
Tufte book (data visualization): amzn.to/3kVrR2o
Few book (data visualization): amzn.to/3n2qTTU
The Hundred-Page Machine Learning Book: amzn.to/3563PfK
Applied Predictive Modeling: amzn.to/38lbf0M
Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: amzn.to/3eAh1N2
Approaching (Almost) Any Machine Learning Problem: amzn.to/36gSjh0
Deep Learning: amzn.to/3p83vVW
A Study Pathway for Data Science: youtu.be/ySI5P37Xf_k
Should You Be a Data Science Generalist or Specialist?: youtu.be/XQOWqSbBPeU
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I revisit the state of the data science job market in 2021 to address the question: is the future of data analytics and data science work remote or hybrid going into the rest of 2021 and beyond? Much of this applies to the broader economic climate beyond data science too. We'll look at a few surveys/studies and articles written on this topic to answer the question for those of you looking for data science jobs in the near future - can you work from home?
Dice: marketing.dice.com/pdf/2021/Workforce_Sentiment_Report.pdf
ComputerWorld: computerworld.com/article/3622356/the-future-of-work-is-hybrid-and-remote.html
Vox: vox.com/recode/22387529/working-from-home-return-to-office-remote-work
WordStream: wordstream.com/blog/ws/2017/06/16/working-remotely
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
2020 report video: youtu.be/Gpel_WehC6s
2021 Anaconda report link: anaconda.com/state-of-data-science-2021
In this video, I want to give my coverage and thoughts on Anaconda's 2021 state of data science report. Anaconda releases this annual report on the state of the data science job market, the participants in the field, and of various technologies at play. It is worth looking at year over year to keep up to date on data science trends.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
In this video, I discuss data engineering. I address the following topics: what exactly data engineering is (we're going to look at a full example), how it differs from data science, will it replace data science, and, of course, how much data engineers are paid and what the future seems to hold for this part of the broader data world.
References:
kdnuggets.com/2021/02/dont-need-data-scientists-need-data-engineers.html
dataquest.io/blog/what-is-a-data-engineer
quanthub.com/what-is-data-engineering
realpython.com/python-data-engineer
xplenty.com/blog/data-engineering-what-does-a-data-engineer-do-how-do-i-become-one
dremio.com/data-lake/data-engineering
medium.com/@rchang/a-beginners-guide-to-data-engineering-part-i-4227c5c457d7
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Topics:
0:00 - Intro
2:10 - Linear regression
7:34 - Logistic regression
8:26 - Supervised & unsupervised learning
9:09 - Inference & prediction
12:02 - Regularization
Referenced RichardOnData videos:
"Inference vs Prediction: An Overview": youtu.be/uh_k1jD35K8
"Interpreting Linear Model Output in R": youtu.be/8hmlbamLB4U
"Training and Tuning ML Models in R": youtu.be/rTphHY9Chgk
In this video, I discuss the practical reasons when and why you should you use regression methods (linear or logistic regression).
Let's start with the premise that we have a dataset where every cell has a value, every row is an observation, and every column is a variable. Then let's suppose we have one variable that represents some kind of "outcome" or "response" that we care about. We'll typically represent that with the letter Y. The remaining variables are interchangeably called terms like predictors, covariates, or explanatory variables, and are represented as the design matrix X or alternately as the X's.
In the simpler case where we have just two variables Y and X, we may begin to understand the relationship between these two variables by creating a scatterplot of them or looking at the correlation coefficient r. A regression model will formalize our understanding of the relationship between these variables - and the concept of "ordinary least squares" (minimizing the sum of the squared residuals, where residuals are the differences between observed and fitted values) can be projected into more than two dimensions.
More broadly speaking, regression methods are a technique typically used for "supervised learning" problems where we do have a defined response variable. Contrast this with "unsupervised learning" problems where we do not - our goal may be dimensionality reduction (e.g. principal component analysis) or defining our own classes or "clusters". In these supervised learning problems, we further have one of two goals: inference (inferring the effect a variable or variables have on the response variable), or prediction. The entire discussion above pertains to inference! It is well known that machine learning methods tend to have stronger predictive performance than statistical methods like regression.
However, regularization provides a way to balance this. There are two popular types: L2 regularization ("ridge regression") or L1 regularization ("lasso regression" or "lasso"). These models will shrink model coefficients; thereby introducing bias in exchange for reducing variance. The key difference is that the lasso can reduce coefficients to zero, thereby doubling as a feature selection approach; this does not occur with ridge regression. This can lead to a reduction in MSE; that is, predictive performance can improve - however, this is much more challenging to interpret and explain to prospective clients.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Caret tutorial series:
Part 1: youtu.be/SYUMTRt70pk
Part 2: youtu.be/9vrf9O5ef6g
Part 3: youtu.be/rTphHY9Chgk
Part 4: youtu.be/epFy6SPIuWs
Tidymodels:
Part 1: youtu.be/CqPnO0wep6Q
Part 2: youtu.be/sjatHUL66sA
In the previous tutorial series, we walked through the "caret" package in R for machine learning. We used the raw "GermanCredit" dataset, performed a brief exploration of it, and used the package to walk through a variety of steps: pre-processing, removing low information features, tuning hyperparameters, correcting for class imbalance, and summarizing results based on metrics we deem important. Where possible, we will perform the exact same exercise here, except we will use the "tidymodels" suite of packages to do so.
There are a few sources from which this tutorial draws influence and structure.
- "Tutorial on tidymodels for Machine Learning": hansjoerg.me/2020/02/09/tidymodels-for-machine-learning
- "Tidymodels: tidy machine learning in R": http://www.rebeccabarter.com/blog/2020-03-25_machine_learning
- "Caret vs. tidymodels - comparing the old and new" by Konrad Semsch: https://konradsemsch.netlify.app/2019/08/caret-vs-tidymodels-comparing-the-old-and-new/
- "Tidy Modeling with R" by Max Kuhn and Julia Silge: tmwr.org
- Recursive feature elimination example by Max Kuhn: github.com/stevenpawley/recipeselectors/issues/1
- Documentation for "stacks": stacks.tidymodels.org/articles/basics.html
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Caret tutorial series:
Part 1: youtu.be/SYUMTRt70pk
Part 2: youtu.be/9vrf9O5ef6g
Part 3: youtu.be/rTphHY9Chgk
Part 4: youtu.be/epFy6SPIuWs
Tidymodels:
Part 1: youtu.be/CqPnO0wep6Q
In the previous tutorial series, we walked through the "caret" package in R for machine learning. We used the raw "GermanCredit" dataset, performed a brief exploration of it, and used the package to walk through a variety of steps: pre-processing, removing low information features, tuning hyperparameters, correcting for class imbalance, and summarizing results based on metrics we deem important. Where possible, we will perform the exact same exercise here, except we will use the "tidymodels" suite of packages to do so.
There are a few sources from which this tutorial draws influence and structure.
- "Tutorial on tidymodels for Machine Learning": hansjoerg.me/2020/02/09/tidymodels-for-machine-learning
- "Tidymodels: tidy machine learning in R": http://www.rebeccabarter.com/blog/2020-03-25_machine_learning
- "Caret vs. tidymodels - comparing the old and new" by Konrad Semsch: https://konradsemsch.netlify.app/2019/08/caret-vs-tidymodels-comparing-the-old-and-new/
- "Tidy Modeling with R" by Max Kuhn and Julia Silge: tmwr.org
- Recursive feature elimination example by Max Kuhn: github.com/stevenpawley/recipeselectors/issues/1
- Documentation for "stacks": stacks.tidymodels.org/articles/basics.html
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Caret tutorial series:
Part 1: youtu.be/SYUMTRt70pk
Part 2: youtu.be/9vrf9O5ef6g
Part 3: youtu.be/rTphHY9Chgk
Part 4: youtu.be/epFy6SPIuWs
In the previous tutorial series, we walked through the "caret" package in R for machine learning. We used the raw "GermanCredit" dataset, performed a brief exploration of it, and used the package to walk through a variety of steps: pre-processing, removing low information features, tuning hyperparameters, correcting for class imbalance, and summarizing results based on metrics we deem important. Where possible, we will perform the exact same exercise here, except we will use the "tidymodels" suite of packages to do so.
There are a few sources from which this tutorial draws influence and structure.
- "Tutorial on tidymodels for Machine Learning": hansjoerg.me/2020/02/09/tidymodels-for-machine-learning
- "Tidymodels: tidy machine learning in R": http://www.rebeccabarter.com/blog/2020-03-25_machine_learning
- "Caret vs. tidymodels - comparing the old and new" by Konrad Semsch: https://konradsemsch.netlify.app/2019/08/caret-vs-tidymodels-comparing-the-old-and-new/
- "Tidy Modeling with R" by Max Kuhn and Julia Silge: tmwr.org
- Recursive feature elimination example by Max Kuhn: github.com/stevenpawley/recipeselectors/issues/1
- Documentation for "stacks": stacks.tidymodels.org/articles/basics.html
Skip ahead:
2:43 - dplyr
3:54 - data.table
5:09 - tidyr
6:12 - ggplot2
7:50 - ggThemeAssist
10:08 - esquisse
13:04 - plotly
14:51 - purrr
16:20 - stringr
16:54 - lubridate
17:44 - forcats
18:59 - rmarkdown
20:24 - kableExtra
21:25 - shiny
22:30 - shinyDashboard
23:17 - caret
24:10 - tidymodels
25:45 - keras
27:03 - fable
28:26 - reticulate
RichardOnData tutorials:
dplyr: youtu.be/6Z48qcrTfYU
ggplot2/ggThemeAssist: youtu.be/2qPQhXnbk3o
tidyr: youtu.be/4_7MDLTWflA
lubridate: youtu.be/VYAo69WdJZg
forcats: youtu.be/6aHrHzHgHms
stringr: youtu.be/cVBvpi4qgc4
kableExtra: youtu.be/JqUViTDoSEo
caret (part 1): youtu.be/SYUMTRt70pk
caret (part 2): youtu.be/9vrf9O5ef6g
caret (part 3): youtu.be/rTphHY9Chgk
caret (part 4): youtu.be/epFy6SPIuWs
#LearnR #RPackages #BreakingIntoDataScience
Reference links:
dplyr: dplyr.tidyverse.org
data.table: cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.html
tidyr: tidyr.tidyverse.org
ggplot2: ggplot2.tidyverse.org
esquisse: rdrr.io/cran/esquisse
plotly: plotly.com/r
purrr: purrr.tidyverse.org
stringr: stringr.tidyverse.org
lubridate: lubridate.tidyverse.org
forcats: forcats.tidyverse.org
rmarkdown: rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf
kableExtra: cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html
shiny: rstudio.com/wp-content/uploads/2015/02/shiny-cheatsheet.pdf
shinyDashboard: rstudio.github.io/shinydashboard/get_started.html
caret: cran.r-project.org/web/packages/caret/vignettes/caret.html
tidymodels: rviews.rstudio.com/2019/06/19/a-gentle-intro-to-tidymodels
keras: tensorflow.rstudio.com/guide/keras
fable: http://fable.tidyverts.org
reticulate: ugoproto.github.io/ugo_r_doc/pdf/reticulate.pdf
"R for Data Science" digital version: r4ds.had.co.nz
"R for Data Science" amazon link: amzn.to/3tNFKVv
"Deep Learning with R": amzn.to/3txjtdA
All logos used are the property of RStudio. I am not the creator of the mentioned logos, packages, or materials.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Part 1: youtu.be/SYUMTRt70pk
Part 2: youtu.be/9vrf9O5ef6g
Part 3: youtu.be/rTphHY9Chgk
In this video I continue the tutorial series on machine learning in R to talk about the following topics: modifying boundaries for classifier thresholds, creating ROC curves, and ensembling the results of multiple models. We will supplement the "caret" package with "ROCR" and "caretEnsemble".
There are a few sources from which this tutorial draws influence and structure. The first is the GitHub documentation on "caret" from its creation, Max Kuhn. The second is a very well-written and comprehensive tutorial by author Selva Prabhakaran on Machine Learning Plus. Third is a helpful resource for dealing with class imbalance, as we often find with classification problems.
- GitHub documentation from Max Kuhn: topepo.github.io/caret
- Tutorial by Selva Prabhakaran: machinelearningplus.com/machine-learning/caret-package
In this video I discuss some of the factors that make machine learning in real life so much different from what you see in books and YouTube. The possibilities are endless and most have to do with the underlying data. Here is a summary:
1) You usually must start by merging different datasets together. If the underlying data are complex enough this can take longer than any model training activity.
2) The sample size can be either very small or very large - both present challenges.
3) Continuous variables can have outliers.
4) Factor variables can also have outliers - that is, levels that occur infrequently.
5) Variables can have lots of missingness. It can raise a real question of whether these values are missing at random or there is meaning behind the madness.
6) Variables can be confusing and it's unclear which ones are meaningful or meaningless.
7) Anything to do with text data can be open-ended and every data scientist will approach it differently!
8) The response variable in classification problems can often be extremely imbalanced.
9) It is not always clear how to determine model performance, or what performance constitutes "good enough".
10) If you have to put the model into production this raises a whole new set of considerations not often discussed in books!
#MachineLearning #RealLifeMachineLearning #BreakingIntoDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Shortcuts:
0:00 - Intro
2:07 - Demand
6:57 - Supply
8:55 - Trends
11:46 - Salaries
"Will Data Science Die in 5 Years?": youtu.be/VQq8DwEJeoU
In this video I discuss the data science job market as it stands in 2021, with an eye towards the outlook following the pandemic. The trend has been that data science has experienced enormous growth in the last several years; is data science still a rising career in 2021? We look at demand for jobs, any proxy we can find for the supply of data scientists, emerging trends in data science, as well as salary expectations in 2021.
Reference links:
datanami.com/2020/11/16/why-data-science-is-still-a-top-job
towardsdatascience.com/is-data-science-still-a-rising-career-in-2021-722281f7074c
lambdaschool.com/the-commons/data-science-job-growth-in-2021-and-beyond
techhub.dice.com/Dice-Q2-Tech-Job-Report.html
the-next-tech.com/future/the-demand-for-data-science-jobs-in-2020
analyticsinsight.net/analytics-insight-predicts-3-million-job-openings-in-data-science-in-2021
kdnuggets.com/2020/09/data-science-analytics-job-trends.html
kdnuggets.com/2021/01/greatlearning-data-science-analytics-career-trends-2021.html
interviewquery.com/blog-data-science-interview-report
reddit.com/r/datascience/comments/klvb55/official_2020_end_of_year_salary_sharing_thread
#DataScience #DataScienceJobs #BreakingIntoDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Part 1: youtu.be/SYUMTRt70pk
Part 2: youtu.be/9vrf9O5ef6g
In this video I continue the tutorial series on machine learning in R to talk about training and tuning models. We covered the bare minimum requirements in the Part 2 video for training a model and then producing a confusion matrix. In this video we will cover the interpretation of the confusion matrix and then move on to tuning the model via specifying a training control, and we will try some other approaches like upsampling and downsampling to address the class imbalance in our training and testing data.
There are a few sources from which this tutorial draws influence and structure. The first is the GitHub documentation on "caret" from its creation, Max Kuhn. The second is a very well-written and comprehensive tutorial by author Selva Prabhakaran on Machine Learning Plus. Third is a helpful resource for dealing with class imbalance, as we often find with classification problems.
- GitHub documentation from Max Kuhn: topepo.github.io/caret
- Tutorial by Selva Prabhakaran: machinelearningplus.com/machine-learning/caret-package
- Tutorial on "caret" with class imbalances: shiring.github.io/machine_learning/2017/04/02/unbalanced
JuliaObserver: juliaobserver.com
JuliaPackages: juliapackages.com
Julia Newsletter (Jan 2021): juliacomputing.com/blog/2021/01/newsletter-january
(Please note that I am not affiliated with or sponsored by Julia Computing in any way; all views are strictly my own.)
In this video I discuss the Julia programming language and the growth that it experienced in the year 2020. For those of you who are totally unfamiliar, Julia is a general purpose programming language (though it was designed for strength in scientific computing) launched in 2012. Julia has many packages that make data science easy, whether that's DataFrames.jl for data manipulation, IJulia.jl for integration with the Jupyter framework, Plots.jl, Gadfly.jl, or VegaLite.jl for data visualization, or ScikitLearn.jl, TensorFlow.jl, or Flux.jl for machine learning.
There are many metrics pointing to Julia's growth over the last year. One of them is a major increase in the TIOBE index from #47 to #29. Other indicators are growth in the number of downloads and packages, almost doubling up to 25 million and over 5,000 respectively. (The latter is important because Julia package growth makes it much easier for organizations to get what they need out of the language!) To be fair, some of this growth may be due to increased demand for scientific computing and research as a result of the 2020 virus. However, everything we are seeing is very promising. Julia can be on track to compete with R and Python as a major player in the data science world in a few years' time!
#DataScience #JuliaProgramming #JuliaComputing
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
"How to Break Into Data Science in 2021": youtu.be/IHxnCzAWz0o
Video #1 of my R tutorial series: youtu.be/6Z48qcrTfYU
In this video I go over my favorite resources for learning R. I always recommend picking one of R or Python and mastering it - so ok, suppose you want to learn R, HOW exactly do you go about doing that? There's a lot of paid courses out there, but unless you really want that certification, there are so many free resources that you don't need that.
"R for Data Science" digital version: r4ds.had.co.nz
"R for Data Science" amazon link: amzn.to/3tNFKVv
Kaggle: kaggle.com
Codecademy: codecademy.com/learn/learn-r
#DataScience #RProgramming #RForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
"How to Break Into Data Science in 2021": youtu.be/IHxnCzAWz0o
All credit goes to Videvo for video clips.
In this video, I discuss some strategies that you can use to make more money. These are shaped by own experiences as a data scientist, but I have a feeling they'll work for you no matter who you are and where you're located!
1) Get some experience before you get your first big job
2) If you want a raise, ask for one!
3) Sometimes, it is appropriate to leave companies
4) Negotiate during the hiring process
5) Build a network and public presence
#DataScience #BreakingIntoDataScience #MakeMoreMoney
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
"How Much Statistics Do You REALLY Need for Data Science?": youtu.be/_tEek2jPn-4
"How to Get a Data Science Job in 2021": youtu.be/OgXNyZIka-Q
"A Study Pathway for Data Science in 2020": youtu.be/ySI5P37Xf_k
"Can You Become a Data Analyst/Scientist With No College Degree?": youtu.be/RSiKo7daQSU
Sorry about the choppy audio towards the end! I've been having on and off microphone problems.
All credit goes to Videvo for video clips.
In this video, I talk about how to break into data science in the year 2021, suppose if you're a complete beginner who's just starting out. This is a lucrative field and as the economy recovers from the illness over the course of the year and many things become increasingly digitized, there remains little doubt that data science will be in enormous demand!
Here are five steps:
1) Receive some form of accreditation, whether that is a college degree, or Coursera or other types of certification
2) Learn the (right) material!
3) Practice your skills. Some ways include: internships, projects at school, or Kaggle competitions
4) Build a portfolio
5) Get that job!
Statistics Coursera courses:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
Books:
"An Introduction to Statistical Learning": amzn.to/3mzrwmf
"The Hundred-Page Machine Learning Book": amzn.to/3mBOSHK
"The Guru's Guide to Transact-SQL": amzn.to/34ven7L
"R for Data Science": amzn.to/3nOHieF
"Python for Data Analysis": amzn.to/37ySAOc
#DataScience #BreakingIntoDataScience #StatisticsForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Subscribe to RichardOnData here: youtube.com/channel/UCKPyg5gsnt6h0aA8EBw3i6A?sub_confirmation=1
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
I strongly recommend watching my tutorial on R Markdown before this one. It can be found here: youtu.be/fzHzCocjASg
In a previous tutorial, we saw R Markdown in action, where multiple things can be created in one location: code, commentary, and output. A point I made towards the end is that formatting tables can be an area requiring lots of focus in and of itself! For example, calling a tibble is going to look very different than when it is passed through the "kable" function from the "knitr" package. In this tutorial we will also explore another package which will facilitate the creation of presentation-worthy tables: "kableExtra".
- A tutorial on "knitr": sachsmc.github.io/knit-git-markr-guide/knitr/knit.html
- "kableExtra" vignette: cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html
- A "formattable" tutorial: displayr.com/formattable
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
"How Much Statistics Do You REALLY Need for Data Science?": youtu.be/_tEek2jPn-4
"Can You Become a Data Analyst/Scientist With No College Degree?": youtu.be/RSiKo7daQSU
365 Data Science - How to Become a Data Scientist: 365datascience.com/become-data-scientist-2020
365 Data Science - Best Degrees to Become a Data Scientist (2021): 365datascience.com/best-degrees-data-scientist
CollegeVine - The List of All U.S. Colleges With a Data Science Major: blog.collegevine.com/the-list-of-all-u-s-colleges-with-a-data-science-major
#DataScience #BreakingIntoDataScience #DegreesForDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
In this video, I go over a key statistical technique for comparing means in three or more groups: Analysis of Variance (ANOVA). This is an important technique because it, unlike conducting pairwise t-tests, fixes the overall Type I error rate.
ANOVA represents a shift away from the concept of t-tests, in the sense that it involves comparison of variation rather than direct comparison of means. It dissects variation into two components: variation BETWEEN groups (represented by the quantity Sum of Squares Groups, sometimes called Sum of Squares Treatment) and variation WITHIN groups (represented by the quantity Sum of Squares Error).
Sum of Squares Groups has degrees of freedom k - 1, where k is the number of groups. Sum of Squares Error has degrees of freedom N - k, where N is the total sample size in all of the groups together. If we divide these quantities by their respective degree of freedom terms, we arrive at quantities Mean Square Groups and Mean Square Error. These can be thought of as weighted quantities representing the variation between and within groups.
The ratio of these two Mean Square Groups / Mean Square Error is equal to the F statistic; a statistic that follows an F-distribution with parameters equal to the two degree of freedom terms. This is a strictly non-negative, right-skewed distribution. At this point, statistical software can be used to calculate a p-value that can be compared to the significance level.
In this video I tackle the question of whether you should be a data science GENERALIST or a data science SPECIALIST. I delve into what exactly these actually are in practice, the advantages and shortcomings associated with each of these, and some considerations you should have as you launch a career in data science.
Some articles used to inspire this video:
"Why you should be a Generalist first, Specialist later as a Data Scientist?" - Admond Lee: towardsdatascience.com/why-you-should-be-a-generalist-first-specialist-later-as-a-data-scientist-f26d687f8c6
"Generalists vs. Specialists in Data Science and Analytics" - Cameron Warren: towardsdatascience.com/generalists-vs-specialists-in-data-science-and-analytics-fe5d8b55f1e6
"Why You Should be a Data Science Generalist - and How to Become One" - Vincent Granville: datasciencecentral.com/profiles/blogs/why-you-should-be-a-data-science-generalist
"Why Data Science Teams Need Generalists, Not Specialists" - O'Reilly Media: medium.com/oreillymedia/why-data-science-teams-need-generalists-not-specialists-955383d0bb1b
"Want a Job in Data Science? Here's Why You Should Specialize" - Dataquest: dataquest.io/blog/job-data-science-specialize
"Generalists Dominate Data Science" - Russell Jurney: kdnuggets.com/2018/02/generalists-dominate-data-science.html
"Who Can Yield Better Outcomes in Data Science: A Generalist Or a Specialist?" - Smriti Srivastava: analyticsinsight.net/who-can-yield-better-outcomes-in-data-science-a-generalist-or-a-specialist
"Why You Shouldn't be a Data Science Generalist" - Jeremie Harris: kdnuggets.com/2018/12/why-shouldnt-data-science-generalist.html
#DataScience #GeneralistOrSpecialist #BreakingIntoDataScience
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
Part 1: youtu.be/SYUMTRt70pk
"When Should You Use Random Forests?": youtube.com/watch?v=tQi7vanY_RU
In this video I continue with the multi-part tutorial series that began in the last video with the "caret" package. We will finish the process of applying transformations to the training set, and applying the same specified transformations to the testing set. We'll then get into algorithmically removing low information features via recursive feature elimination, training our first model, and then creating variable importance plots where applicable.
There are a few sources from which this tutorial draws influence and structure. The first is the GitHub documentation on "caret" from its creation, Max Kuhn. The second is a very well-written and comprehensive tutorial by author Selva Prabhakaran on Machine Learning Plus. Third is a helpful resource for dealing with class imbalance, as we often find with classification problems.
- GitHub documentation from Max Kuhn: topepo.github.io/caret
- Tutorial by Selva Prabhakaran: machinelearningplus.com/machine-learning/caret-package
- Tutorial on "caret" with class imbalances: shiring.github.io/machine_learning/2017/04/02/unbalanced
Patreon: patreon.com/richardondata
JavaScript for Data Science: js4ds.org/#s:intro
W3Schools tutorial: w3schools.com/js/DEFAULT.asp
Codecademy tutorial: codecademy.com/learn/introduction-to-javascript
D3 galleries:
- d3-graph-gallery.com
- observablehq.com/@d3/gallery
- github.com/d3/d3/wiki/Gallery
In this video I discuss JavaScript as it stands in the year 2021, as it relates to data science. We answer the question - if you're a coming, aspiring data scientist, is this something you should learn?
JavaScript has been around since 1995, and is one of the primary scripting languages of the internet alongside HTML and CSS. The trademark of it is owned by Oracle, but the language is defined by an organization called the European Computer Manufacturers Association (ECMA). It is an object oriented language that is generally compiled "just in time", is dynamically typed, but is not open source. The most common way it is used is through Node, which is a command-line interpreter. Through that you can use NPM, or the Node Package Manager, to install JavaScript libraries.
Some helpful libraries for data science include: D3.js, nlp.js, ml.jl, and TensorFlow.js.
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
In this video I provide a beginning to a multi-part tutorial series on machine learning in R using the "caret" package. We will begin with pre-processing of a dataset to get it into a format appropriate for the machine learning pipeline, as well as identifying zero or near zero variance predictors. The beauty of this package is that it is truly a one stop shop for all of your machine learning needs.
There are a few sources from which this tutorial draws influence and structure. The first is the GitHub documentation on "caret" from its creation, Max Kuhn. The second is a very well-written and comprehensive tutorial by author Selva Prabhakaran on Machine Learning Plus. Third is a helpful resource for dealing with class imbalance, as we often find with classification problems.
- GitHub documentation from Max Kuhn: topepo.github.io/caret
- Tutorial by Selva Prabhakaran: machinelearningplus.com/machine-learning/caret-package
- Tutorial on "caret" with class imbalances: shiring.github.io/machine_learning/2017/04/02/unbalanced
Patreon: patreon.com/richardondata
Can Data Science Predict the Stock Market: youtu.be/y5hh2xqFW4k
In this video I discuss whether or not data science methods can be used to predict the price of bitcoin and/or other cryptocurrencies. When I did a video on the stock market, my conclusion was essentially that while ML/AI methods can help significantly better than random guessing, ultimately you run into three problems:
1) The efficient market hypothesis - the idea that all the information about a stock is already priced in
2) It is hard to predict future shocks
3) There is a conflict of interest in releasing such things
We are going to address the same question as it pertains to bitcoin and cryptocurrencies as a whole.
Links referenced in this video:
Marco Santos - "Predict Bitcoin Prices with Deep Learning": towardsdatascience.com/predicting-bitcoin-prices-with-deep-learning-438bc3cf9a6f
Derk Zomer - "Using machine learning to predict future bitcoin prices":
towardsdatascience.com/using-machine-learning-to-predict-future-bitcoin-prices-6637e7bfa58f
Marco Santos - "Predicting Prices of Bitcoin with Machine Learning":
towardsdatascience.com/predicting-prices-of-bitcoin-with-machine-learning-3e83bb4dd35f
Marie Huillet - "Data Scientist Uses Deep Learning to Predict BTC Price in Real-Time": cointelegraph.com/news/data-scientist-uses-deep-learning-to-predict-btc-price-in-real-time
S M Raju, Ali Mohammad Tarif - "Real-Time Prediction of BITCOIN Price Using Machine Learning Techniques and Public Sentiment Analysis":
arxiv.org/ftp/arxiv/papers/2006/2006.14473.pdf
Rafael Schultze-Kraft - "Don't be fooled -- Deceptive Cryptocurrency Price Predictions Using Deep Learning": hackernoon.com/dont-be-fooled-deceptive-cryptocurrency-price-predictions-using-deep-learning-bf27e4837151
Alex Lielacher - "3 Bitcoin Sentiment Analysis Tools: How They Work & How to Use Them": cryptonews.com/exclusives/top-3-btc-sentiment-analysis-tools-how-they-work-how-to-use-6889.htm
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
0:00 Intro
2:21 App #1
9:35 App #2
12:34 App #3
20:04 Cheat sheet
Shiny - Gallery: shiny.rstudio.com/gallery
Kmeans example: shiny.rstudio.com/gallery/kmeans-example.html
Single-file example: shiny.rstudio.com/gallery/single-file-shiny-app.html
Word cloud: shiny.rstudio.com/gallery/word-cloud.html
Cheat sheet: rstudio.com/wp-content/uploads/2015/02/shiny-cheatsheet.pdf
In this video, I go under the hood to look at THREE different Shiny apps from the RStudio shiny gallery. We will look at the UI and server components of these apps to see how it all comes together to produce the actual apps.
If you do not understand everything at first, that is ok. There is a learning curve to Shiny. The most important things to understand are the idea that the apps are divided into two distinct components: UI and server, and that the server functions through “reactive” functions.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
Video on study pathway for data science: youtu.be/ySI5P37Xf_k
In this video I discuss how you can become a data analyst or data scientist without a college degree. It is more than possible to become a data professional with no degree - however, it will require learning the particular necessary skillsets as well as demonstrating some expertise with those skillsets, somehow.
Statistics Coursera courses:
Duke: coursera.org/specializations/statistics
John Hopkins: coursera.org/specializations/jhu-data-science
University of Amsterdam: coursera.org/specializations/social-science
Books:
"An Introduction to Statistical Learning": amzn.to/3mzrwmf
"The Hundred-Page Machine Learning Book": amzn.to/3mBOSHK
"The Guru's Guide to Transact-SQL": amzn.to/34ven7L
"R for Data Science": amzn.to/3nOHieF
"Python for Data Analysis": amzn.to/37ySAOc
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
In this tutorial, we will provide an introduction to R Markdown. This is an amazing framework for creating fully reproducible reports while leveraging the power of the R programming language. The beauty of R Markdown is that you can write code, perform analysis, and view output in it. It can then be rendered into a document that can be shared with your audience of choice.
An excellent resource for R Markdown, for which this tutorial takes partial influence from, is Chapter 27 of the book "R for Data Science" by Hadley Wickham and Garrett Grolemund.
Additionally, we will be referencing the cheat sheet for R Markdown, which can be located here: rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf
Mathematics in R Markdown: rpruim.github.io/s341/S19/from-class/MathinRmd.html
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
GitHub: github.com/RichardOnData/YouTube-Scripts
PREVIOUS tutorial on creating linear/logistic regression model output: youtu.be/uXmmskpcasc
Inference vs. Prediction video: youtu.be/uh_k1jD35K8
In the previous tutorial we went over creating linear and logistic regression models in R. In this tutorial we will interpreting the output.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32
Patreon: patreon.com/richardondata
Thank you for bringing me to 5,000 subscribers! I want to talk about why I started this channel in the first place and my plans to complete building this channel out as a "one stop shop" for data science in 2021.
PayPal: richardondata@gmail.com
Patreon: patreon.com/richardondata
BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL
ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8
LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32


