Increased compute power discovers opportunities from nationwide datasets

tree in the field

Pairing with HPC Midlands and experts at Karlsruhe Institute of Technology, German (KIT) allowed Bluesky International to turn the challenge of nationwide datasets into an opportunity.

Challenge

The collection of geodata is spreading. While new collection devices, such as smart phones, proliferate the older tools of satellite and aerial-based imagery have advanced to capture more detailed data with applicability to more users.

East Midlands’ based Bluesky International has a decade’s experience harnessing these technologies to create advanced Geographic Information System (GIS) and Computer Aided Design (CAD) datasets that address location-based challenges like air quality, tree stocks and building heat emissions. With Bluesky’s datasets challenges become opportunities, but more data means looking at a bigger scale.

For example, through aerial imagery planes can generate copious amount of light detection and ranging (LIDAR) data on almost any location at any time of year. This data covers entire countries but is most useful when it can provide the minutest detail.

Solution

Working with shadow analysis experts from KIT Bluesky wanted to see how current desktop-based workflows used for small regions could be scaled up.

The team developed an internal open-source shadow analysis programme that detailed 3D surface structures with a ground sampling distance of 25cm2 – roughly the shadow area cast by a single Magpie. For this detail a country like Great Britain has 3.5 trillion single elevation points – a big-data problem unapproachable on conventional machines.

With access to HPC Midlands compute power the project found multiple solutions for creating actionable information from the mass of points.

“We found that we can scale up available workflows relatively easily through a ‘divide and conquer’ approach made available from parallel programming,” said Simon Schuffert, a Research Associate at KIT. “We discovered that a supercomputer fits perfectly with the needs of industry partners who process huge amounts of geodata.”

Impact

The dataset created by a thorough shadow-analysis can be used to accurately predict effectiveness of solar-panels as small as those attached to parking ticket machines, or monthly and annual sun exposure for agricultural areas, among other uses.

“Processing huge amount of nationwide data is challenging,” said Simon. “This big-data needs to be processed in due course to save time and money.”