Data Intelligence Day Dubai 2024 Databricks 2024-06-03 | Data Intelligence Day landed in Dubai on 21 May 2024 and gathered 250 Data & AI experts, to discover the new unified platform that delivers AI for your data, and data for your AI: the Databricks Data Intelligence Platform.Discover Databricks: databricks.com
Simplifying Time Zones with Timestamp_NTZ in Apache Spark Databricks 2024-06-03 | Timestamp_NTZ: docs.databricks.com/en/sql/language-manual/data-types/timestamp-ntz-type.htmlconvert_timezone(): docs.databricks.com/en/sql/language-manual/functions/convert_timezone.html
A (brief) deep dive into Delta for data storage Databricks 2024-05-29 | Links to docs:Open Source: delta.io/learn/getting-startedBeginners Tutorial: docs.databricks.com/en/delta/tutorial.html#language-sql
Databricks New AI Powered Visualization Tool Databricks 2024-05-27 | Want to try lakeview? Check out this tutorial: docs.databricks.com/en/dashboards/create-a-dashboard.html
Isolate the Lakehouse: Deploying an isolated architecture on Databricks Databricks 2024-05-23 | In this video, we will talk about isolating the lakehouse and how to deploy an isolated architecture on Databricks - what we call an "Isolake". This architecture is ideal for highly related industries or environments for non-internal users who may need access to a subset of your data in Databricks. This video will walk you through what an isolated architecture is and its various features and limitations of an isolated architecture, and a demo on how to set one up. For a full explanation, check out our blog post: medium.com/databricks-platform-sme/isolake-a-simplistic-deployment-design-to-an-isolated-databricks-lakehouse-on-aws-c0f98b5bbba0Isolake GitHub: github.com/JDBraun/isolake AWS Security Reference architecture: github.com/databricks/terraform-databricks-sra
Turn your strings into SQL with Execute Immediate Databricks 2024-05-22 | Serge Reilau's blog: community.databricks.com/t5/technical-blog/building-sql-with-sql-an-introduction-to-execute-immediate/ba-p/59620Docs: docs.databricks.com/en/sql/language-manual/sql-ref-syntax-aux-execute-immediate.html
Demo: Shutterstock on Databricks Marketplace Databricks 2024-05-20 | Watch the demo to learn more about Shutterstock’s listing on Databricks Marketplace, how to access it and query it using a notebook.Read this blog to learn more about Shutterstock Listing: databricks.com/blog/shutterstocks-image-datasets-now-databricks-marketplace
Make your Databricks SQL more dynamic with Variables and Identifier() Databricks 2024-05-20 | See the full syntax here:Variables docs.databricks.com/en/sql/language-manual/sql-ref-variables.htmlIdentifier docs.databricks.com/en/sql/language-manual/sql-ref-names-identifier-clause.html
Getting started as a Private Exchange provider on Databricks Marketplace Databricks 2024-05-16 | This demo video walks through the process of signing up to be a private exchange provider on Databricks Marketplace. Companies can now become Private Exchange providers without having to go through the Databricks Marketplace public provider onboarding process.Databricks Marketplace: marketplace.databricks.com
Databricks Marketplace: How Private Exchanges work? Databricks 2024-05-16 | Check out this demo video detailing how private exchanges on Databricks Marketplace work and how they can be used for inter-org collaboration.Databricks Marketplace: marketplace.databricks.com
Never lose another data source again with Databricks Lineage Databricks 2024-05-15 | To find out more details, check out the docs here:docs.databricks.com/en/data-governance/unity-catalog/data-lineage.html
Monitor Databricks Usage like a pro with System Tables Databricks 2024-05-13 | System Tables Docs: docs.databricks.com/en/administration-guide/system-tables/index.html
Data Intelligence Day Hong Kong 2024 Databricks 2024-05-10 | Discover the new unified platform that delivers AI for your data, and data for your AI: the Databricks Data Intelligence Platform.
How Databricks built the LLM DBRX and how you can build yours Databricks 2024-05-10 | To read more about DBRX check out the blog here: databricks.com/blog/introducing-dbrx-new-state-art-open-llmWant to start using it? Find it either in Databricks' AI playground or on HuggingFace here: huggingface.co/databricks/dbrx-instruct
Streaming Tables and Materialized Views on Databricks SQL Databricks 2024-05-09 | In this video, you will learn how to ingest your data using streaming tables and transforming them using materialized views on Databricks SQL. With streaming tables and materialized views, SQL analysts are now empowered to perform data engineering tasks and introduce real-time capabilities within their existing workflow.
Building the Best Models with the Help from AutoML Databricks 2024-05-06 | Check out the docs here: docs.databricks.com/en/machine-learning/automl/index.html
XML Data Ingestion with Spark on Databricks Databricks 2024-05-03 | In this video, we'll explore how to process XML data efficiently using Apache Spark on the Databricks platform. XML is a widely used format for storing and exchanging structured data, and Spark provides powerful capabilities for working with XML files.We'll cover various aspects of XML processing, including reading XML data into Spark DataFrames, validating against XSD schemas, handling schema evolution with Auto Loader, using SQL with XML, and leveraging built-in SQL XML functions. By the end of this video, you'll have a comprehensive understanding of how to work with XML data in Spark on Databricks, empowering you to tackle real-world XML processing tasks with confidence.
5 Reasons Delta Live Tables is my guilty pleasure Databricks 2024-05-03 | Check out our getting started guide here: docs.databricks.com/en/delta-live-tables/tutorial-pipelines.htmlIf you don't have access to Databricks to try it out, check out thins link here: databricks.com/discover/pages/getting-started-with-delta-live-tables
Next Generation Dashboards on Databricks SQL Databricks 2024-05-01 | In this video you will learn about our latest improvements to our next generation dashboards on Databricks SQL. These dashboards feature an enhanced visualization library and a new configuration experience so that you can streamline the process of transforming data into useful insights. Leveraging AI you can create visualizations using natural language and this can be helpful for technical or non-technical users who would like to derive insights from their datasets.
Getting Started on Databricks SQL for Administrators Databricks 2024-04-30 | In this video, you will learn about how to get started using our Data Warehousing solution on Databricks called Databricks SQL from an administrator perspective.
Easy Real Time Model Serving with Databricks Databricks 2024-04-29 | Docs to get started: docs.databricks.com/en/machine-learning/model-serving/index.html
MLflow Registry: Share, Control, and Version Your Machine Learning Models with Ease #datascience Databricks 2024-04-26 | If you're using Unity Catalog docs.databricks.com/en/machine-learning/manage-model-lifecycle/index.htmlIf you're not docs.databricks.com/en/machine-learning/manage-model-lifecycle/workspace-model-registry.html
Never lose a model again with MLflow Experiment Tracking Databricks 2024-04-24 | Quickstart guide docs.databricks.com/en/mlflow/quick-start-python.htmlWhat is MLflow? docs.databricks.com/en/mlflow/index.html
Data Intelligence Day Manila 2024 Databricks 2024-04-22 | Data Intelligence Day Manila took place on 18 April 2024 and gathered over 600 industry leaders and data and AI experts.
MLflow: The Ultimate Tool for Building and Deploying AI Models #datascience #data Databricks 2024-04-22 | Databricks Docs: docs.databricks.com/en/mlflow/index.htmlOpen Source Docs: mlflow.org/docs/latest/introduction/index.html
Serverless for Notebooks - Part 2 #databricks #python Databricks 2024-04-19 | Link to docs: docs.databricks.com/en/compute/serverless.html
State Reader API Demo: Querying State Data in Spark Structured Streaming Databricks 2024-04-15 | The State Reader API allows users to access and analyze Structured Streaming's internal state data. Its primary purpose is facilitating the development, debugging, and troubleshooting of stateful Structured Streaming workloads.The State Reader API introduces much-needed transparency, accessibility, and ease of use to stateful streaming processes. It is straightforward and user-friendly, simplifying complex investigative tasks.View the other demos on the Databricks Demo Hub: databricks.com/resources/demos?utm_source=youtube&utm_medium=web&utm_campaign=7013f000000cVKYAA2
How to save thousands on your Databricks Jobs Databricks 2024-04-15 | Docs: docs.databricks.com/en/workflows/jobs/create-run-jobs.html#use-shared-job-clusters
How to be efficient processing arrays Databricks 2024-04-12 | Check out Serge's blog: community.databricks.com/t5/technical-blog/lambda-functions-knights-of-the-higher-order/ba-p/61264Databricks Docs: docs.databricks.com/en/optimizations/higher-order-lambda-functions.htmlOpen Source Spark docs: spark.apache.org/docs/latest/api/python/reference/api/pyspark.RDD.aggregate.html
The easiest way to stream your data into Delta Databricks 2024-04-10 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/ingestion/auto-loader/file-detection-modes.html
Unlock the power of Data Sharing with Databricks Marketplace #data #machinelearning #databricks Databricks 2024-04-08 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/marketplace/index.html
See Databricks Assistant in Action! Databricks 2024-04-05 | Explore the future of data intelligence with the Databricks Assistant, your context-aware AI assistant. Powered by DatabricksIQ, the Data Intelligence Engine for the Databricks Platform, the Assistant integrates seamlessly into numerous authoring environments within your Workspace— including Notebooks, the SQL editor, Help, and the Lakeview dashboards for text-to-visualization experiences. For more information about Databricks Assistant and DatabricksIQ:► Read the blog: databricks.com/blog/introducing-databricks-assistant ► Visit the Databricks Assistant web page: databricks.com/product/databricks-assistant► Check out the DatabricksIQ web page: databricks.com/product/databricksiqEnjoy!
Supercharge Your Clusters with Databricks Photon #data #cloud #databricks Databricks 2024-04-05 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Want a more in depth explanation? Check out my conference talk here: youtube.com/watch?v=kL2XWgLeXHE
The Data Intelligence Platform for Energy Databricks 2024-04-04 | Energy’s first unified platform brings the power of AI to your data and people, so you can deliver AI’s potential to every energy solution.Explore the platform- databricks.com/solutions/industries/oil-and-gas
Intelligent Data Warehousing on Azure Lakehouse with Databricks Data Intelligence Platform Databricks 2024-04-04 | Intelligent Data Warehousing on Azure Lakehouse with Databricks Data Intelligence Platform talk re-recording from the Microsoft Fabric Community Conference.View demos of Databricks SQL: youtube.com/playlist?list=PLTPXxbhUt-YXXXlfUPTmlpJ7jvw9KTGAv
Getting Started on Databricks SQL for Users Databricks 2024-04-04 | In this video, you will learn how to leverage an end-to-end Data Warehousing and analytics solution right here on Databricks SQL. You will also learn how we have optimized our Data Intelligence platform using artificial intelligence, natural language processing and large language models.00:00 - 00:48: Introduction to Databricks SQL Reference Architecture00:49 - 03:14: Configuring our Compute Resource, the Serverless SQL Warehouse03:15 - 04:40: Navigating the Catalog Explorer04:41 - 04:50: Introduction to the SQL Editor04:51 - 05:34: Creating Streaming Tables05:35 - 07:01: Creating Materialized Views and using the Databricks Assistant07:02 - 08:46: Performing Data Modeling and using the Entity Relationship Diagram08:47 - 09:30: Query History and Query Profile09:31 - 12:57: Creating a Lakeview Dashboard12:58 - 13:44: Integrating with Power BI13:45 - 14:28: Configuring Alerts for your data14:29 - 15:24: Orchestrating your data using Workflows15:25 - 16:03: Analyzing lineage between your data assets16:04 - 17:45: Leveraging LLMs by using AI Functions in SQL
Lightspeed fast improvement for Apache Spark #data #realtime #streaming Databricks 2024-04-03 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: databricks.com/blog/performance-improvements-stateful-pipelines-apache-spark-structured-streaming
Optimization Secrets: Maximizing Efficiency in Data Pipelines #data #cloud #finops #databricks Databricks 2024-04-01 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: databricks.com/discover/pages/optimize-data-workloads-guide
How to schedule your ETL for free #data #cloud #ETL Databricks 2024-03-29 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/workflows/index.html
Databricks is powering NAB’s data strategy for 8.5 million customers Databricks 2024-03-28 | NAB, Australia’s largest business bank, relies on Databricks to securely deliver data at speed and scale. Databricks provides access to reliable data in one platform, and Unity Catalog enables governance across the company – ensuring access to data whenever business users need it. NAB is unlocking use cases that were previously out of reach. Now, they can explore generative AI for customer service, marketing campaigns and financial crime detection, reporting and monitoring.✔️ Hear more from our customers: databricks.com/customers✔️ Read the Databricks blog: databricks.com/blog/category/company/customers✔️ Learn more about NAB: https://www.nab.com.au/👉 Get Social LinkedIn ► linkedin.com/company/databricksInstagram ► instagram.com/databricksincTwitter ► twitter.com/databricks#datagovernance #dataintelligence
DBRX: A New Standard for Open Source LLMs Databricks 2024-03-27 | Today we're excited to announce our new open source general-purpose language model - DBRX. DBRX sets a new standard for efficient LLMs. It outperforms all existing open source models as well as GPT 3.5 on standard benchmarks.DBRX plus Mosaic AI helps developers deploy production-quality GenAI applications that are safe, accurate, and governed. Since it’s fully open source, users can further augment or fine-tune the model and even build their own custom LLM using DBRX as a reference design.Read more here: databricks.com/blog/announcing-dbrx-new-standard-efficient-open-source-customizable-llms
How to share cloud data for free #data #collaboration #databricks #cloudflare Databricks 2024-03-27 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/data-sharing/manage-egress.html#r2
OLMo: Open Language Model [Webinar + Q&A] Databricks 2024-03-27 | LIVE NOW 🚨 The Allen Institute for AI (AI2) has released OLMo 7B, a truly open, state-of-the-art large language model, alongside the pre-training data and training code. This empowers researchers and developers to use the best and open models to advance the science of language models collectively. Interested in learning more about OLMo and how you can use it? Join us for a conversation with Akshita Bhagia, Senior Research Engineer at AI2 and one of the lead researchers on OLMo, and Daniel Liden and Kobie Crawford from Databricks to dive deeper into OLMo.
DatabricksIQ Discussion and Demo: Adding Intelligence to Databricks Search Databricks 2024-03-25 | We discuss the recent additions to Databricks’ in-product search. Topics include the new full-page UI, popularity boosting, semantic search, natural language query understanding, DatabricksIQ, and more!► Hosted by Ari Kaplan from Databricks► Guest Richard Tomlinson from Databricks.For more information about Databricks Search:1. Read the blog: databricks.com/blog/adding-intelligence-to-databricks-search 2. Check out the product docs: docs.databricks.com/en/databricksiq/index.html
5 Databricks Runtime Mistakes I See Happen Way Too Often #data #databricks #cloud Databricks 2024-03-25 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them.Docs: docs.databricks.com/en/release-notes/runtime/index.html
Making the boring bits of Machine Learning easier #MachineLearning #DataScience #Data #Databricks Databricks 2024-03-25 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/machine-learning/feature-store/index.html
Databricks uses AI to make search much faster #AI #Data #Databricks #UnityCatalog Databricks 2024-03-25 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them.
How to use Large Language Models in Databricks SQL #ai #llm #databricks #sql Databricks 2024-03-25 | I'm Holly Smith and I post 3 times a week to the Databricks main channel. Subscribe if you don't want to miss them. Docs: docs.databricks.com/en/large-language-models/ai-functions.html
How to compare GenAI models in Databricks #ai #llm #databricks Databricks 2024-03-25 | docs.databricks.com/en/large-language-models/ai-playground.html
