Visualizations in Qubole Notebooks are not limited to the graphing functions available out of the box. In order to use third-party visualizations, we’ll need to go back into the Spark Qubole Notebook and select the relevant libraries we want to import from our packages. In this section, we will:
Import Pandas and libraries for plotting
Use Pandas DataFrame
Advanced Visualization with Maps
Using Different Packages and Libraries with Apache Spark
The example libraries used are all contained in this Earthquake Visualization Notebook (MatPlotLib, Plotly, and Folium Maps). In order to use some of these more advanced visualizations, we’ll need to import our Pandas library by converting our Spark DataFrame into a Pandas DataFrame*, which has more features than just Spark alone.
First we’ll create and visualize the year_count table. Given we’re using Python here we’ll also need to initialize it with %PySpark