CASE STUDY—A BIG-DATA MOBILE APPLICATION
Google’s Waze GPS navigation app, with its 90 million monthly active users, is one of the most successful bigdata apps. Early GPS navigation devices and apps relied on static maps and GPS coordinates to determine the best route to your destination. They could not adjust dynamically to changing traffic situations.
Waze processes massive amounts of crowdsourced data—that is, the data that’s continuously supplied by their users and their users’ devices worldwide. They analyze this data as it arrives to determine the best route to get you to your destination in the least amount of time. To accomplish this, Waze relies on your smartphone’s Internet connection. The app automatically sends location updates to their servers (assuming you allow it to). They use that data to dynamically reroute you based on current traffic conditions and to tune their maps. Users report other information, such as roadblocks, construction, obstacles, vehicles in breakdown lanes, police locations, gas prices and more. Waze then alerts other drivers in those locations. Waze uses many technologies to provide its services. We’re not privy to how Waze is implemented, but we infer below a list of technologies they probably use. You’ll use many of these in Chapters 11–16. For example,
- Most apps created today use at least some opensource software. You’ll take advantage of many opensource libraries and tools through-out this article.
- Waze communicates information over the Internet between their servers and their users’ mobile devices. Today, such data often is transmitted in JSON (JavaScript Object Notation) format, which we’ll introduce in Chapter 9 and use in subsequent chapters. The JSON data is typically hidden from you by the libraries you use.
- Waze uses speech synthesis to speak driving directions and alerts to you, and speech recognition to understand your spoken commands. We use IBM Watson’s speech-synthesis and speechrecognition capabilities in Chapter 13.
- Once Waze converts a spoken naturallanguage command to text, it must determine the correct action to perform, which requires natural language processing (NLP). We present NLP in Chapter 11 and use it in several subsequent chapters.
- Waze displays dynamically updated visualizations such as alerts and maps. Waze also enables you to interact with the maps by moving them or zooming in and out. We create dynamic visualizations with Matplotlib and Seaborn through-out the article, and we display interactive maps with Folium in Chapters 12 and 16.
- Waze uses your phone as a streaming Internet of Things (IoT) device. Each phone is a GPS sensor that continuously streams data over the Internet to Waze. In Chapter 16, we introduce IoT and work with simulated IoT streaming sensors.
- Waze receives IoT streams from millions of phones at once. It must process, store and analyze that data immediately to update your device’s maps, to display and speak relevant alerts and possibly to update your driving directions. This requires massively parallel processing capabilities implemented with clusters of computers in the cloud. In Chapter 16, we’ll introduce various bigdata infrastructure technologies for receiving streaming data, storing that big data in appropriate databases and processing the data with software and hardware that provide massively parallel processing capabilities.
- Waze uses artificialintelligence capabilities to perform the dataanalysis tasks that enable it to predict the best routes based on the information it receives. In Chapters 14 and 15 we use machine learning and deep learning, respectively, to analyze massive amounts of data and make predictions based on that data.
- Waze probably stores its routing information in a graph database. Such databases can efficiently calculate shortest routes. We introduce graph databases, such as Neo4J, in Chapter 16.
- Many cars are now equipped with devices that enable them to “see” cars and obstacles around them. These are used, for example, to help implement automated braking systems and are a key part of selfdriving car technology. Rather than relying on users to report obstacles and stopped cars on the side of the road, navigation apps could take advantage of cameras and other sensors by using deeplearning computervision techniques to analyze images “on the fly” and automatically report those items. We introduce deep learning for computer vision in Chapter 15.