All the data you need.

Tag: Spark

Reputation Risk: Improving Business Competency and Nurturing Happy Customers by Building a Risk Analysis Engine
Why reputation risk matters? When it comes to the term “risk management”, Financial Service Institutions (FSI) have seen guidance and... The post Reputation Risk: Improving Business Competency and Nurturing Happy Customers by Building a Risk Analysis Engine appeared first on Databricks.
Faster SQL: Adaptive Query Execution in Databricks
Earlier this year, Databricks wrote a blog on the whole new Adaptive Query Execution framework in Spark 3.0 and Databricks... The post Faster SQL: Adaptive Query Execution in Databricks appeared first on Databricks.
Announcing Single-Node Clusters on Databricks
Databricks is used by data teams to solve the world’s toughest problems. This can involve running large-scale data processing jobs... The post Announcing Single-Node Clusters on Databricks appeared first on Databricks.
Integrating large-scale Genomic Variation and Annotation Data with Glow
Genomic annotations augment variant data by providing context for each change in the genome. For example, annotations help answer questions... The post Integrating large-scale Genomic Variation and Annotation Data with Glow appeared first on Databricks.
Analyzing Algorand Blockchain Data with Databricks Delta
Algorand is a public, decentralized blockchain system that uses a proof of stake consensus protocol. It is fast and energy-efficient,... The post Analyzing Algorand Blockchain Data with Databricks Delta appeared first on Databricks.
Flipp Presents Their Lakehouse Architecture with Delta Lake at Tableau Conference
Databricks at the Tableau Conference 2020 Our session Our page The Tableau Conference 2020 begins tomorrow, with our session Databricks:... The post Flipp Presents Their Lakehouse Architecture with Delta Lake at Tableau Conference appeared first on Databricks.
Measuring Advertising Effectiveness with Sales Forecasting and Attributing
Click below to download the notebooks for this solution accelerator: Campaign Effectiveness — ETL Campaign Effectiveness — Machine Learning How... The post Measuring Advertising Effectiveness with Sales Forecasting and Attributing appeared first on Databricks.
Detecting Criminals and Nation States through DNS Analytics
You are a security practitioner, a data scientist or a security data engineer; you’ve seen the Large Scale Threat Detection... The post Detecting Criminals and Nation States through DNS Analytics appeared first on Databricks.
Data + AI Summit Europe Goes Virtual With a Data-Centric Agenda
Technical conferences evolve over time. They expand beyond their initial focus, adding new technologies, attracting new attendees and broadening their... The post Data + AI Summit Europe Goes Virtual With a Data-Centric Agenda appeared first on Databricks.
Diving Into Delta Lake: DML Internals (Update, Delete, Merge)
Diving Into Delta Lake: DML Internals (Update, Delete, Merge) In previous blogs Diving Into Delta Lake: Unpacking The Transaction Log... The post Diving Into Delta Lake: DML Internals (Update, Delete, Merge) appeared first on Databricks.
Registration Open for Inaugural Data + AI Summit Europe
Data + AI Summit Europe will take place 17-19 November and is now open for registration! Formerly known as the... The post Registration Open for Inaugural Data + AI Summit Europe appeared first on Databricks.
Building a Modern Risk Management Platform in Financial Services
This blog was collaboratively written with Databricks partner Avanade. A special thanks to Dael Williamson, Avanade CTO, for his contributions. Financial Institutions today are still struggling to keep up with the emerging risks and threats facing their business. Managing risk, especially within the banking sector, has increased in complexity over …
Easily Clone your Delta Lake for Testing, Sharing, and ML Reproducibility
Introducing Clones An efficient way to make copies of large datasets for testing, sharing and reproducing ML experiments We are excited to introduce a new capability in Databricks Delta Lake – table cloning. Creating copies of tables in a data lake or data warehouse has several practical uses. However, given …
It’s an ESG World and We’re Just Living in it
The future of finance goes hand in hand with socially responsible investing, environmental stewardship, and corporate ethics. In order to stay competitive, Financial Services Institutions (FSI) are increasingly disclosing more information about their environmental, social, and corporate governance (ESG) performance. Hence the increasing importance of ESG ratings and ESG scores …
An Update on Project Zen: Improving Apache Spark for Python Users
Apache Spark™ has reached its 10th anniversary with Apache Spark 3.0 which has many significant improvements and new features including but not limited to type hint support in pandas UDF, better error handling in UDFs, and Spark SQL adaptive query execution. It has grown to be one of the most …
Spark + AI Summit Europe is Expanding and Getting a New Name: Data + AI Summit Europe
Back in 2013, we held the first Spark Summit — a gathering of the Apache Spark™ community with leading contributors and production users sharing their wisdom. Since the first event, Spark’s success has accelerated the evolution of data science, data engineering and analytics. As the data community has expanded, we’ve …
Enabling Spark SQL DDL and DML in Delta Lake on Apache Spark 3.0
Last week, we had a fun Delta Lake 0.7.0 + Apache Spark 3.0 AMA where Burak Yavuz, Tathagata Das, and Denny Lee provided a recap of Delta Lake 0.7.0 and answered your Delta Lake questions. The theme for this AMA was the release of Delta Lake 0.7.0 coincided with the …
A Guide to the Databricks + AWS Cloud Data Lake Dev Day Workshop
The Databricks team has been working hard to recreate content and enhance the experience as we transition all our events to a virtual experience. And we’ve learned a thing or two about what you want to learn: integrating Databricks with AWS services. So we’re excited to present our Cloud Data …