Hello Future !
We have been experimenting parallel processes slow down in the use of your package with lidar processing (airborne lidar and lidR R package).
The issue is also accessible on the lidR github page: r-lidar/lidR#835
Basically the idea is to use future to parallel point cloud processing tasks.
It is very useful to perform complete point cloud processing workflow on large datasets and for a large community.
For example, we often use following codes:
future::plan(future::multisession, workers = params_general$n_cores)
cat("lidR::rasterize_terrain()")
dtm <- rasterize_terrain(ctg, res = resolution, algorithm = tin())
dtm_tiles <- list.files(file.path(params_general$path_data, name_raster), pattern = "_dtm_nooverhangs\\.tif$", full.names = TRUE)
dtm = terra::vrt(dtm_tiles)
terra::writeRaster(dtm,
filename = file.path(path_products, paste0(name_raster, ".tif")),
overwrite = TRUE)
future::plan(future::sequential)
The code uses both future and lidR to create a raster of the terrain topography, tile-by-tile and in parallel thanks to future.
At first, the right number of cores is used (n_cores, checked using htop) but with time, this number tends to decrease and reaching the end it is very low (sometimes only 1 core used)
The issue seems to come from future since the lidR code has not changed for a lot of time.
Do you already know about this limitation ?
Hello Future !
We have been experimenting parallel processes slow down in the use of your package with lidar processing (airborne lidar and lidR R package).
The issue is also accessible on the lidR github page: r-lidar/lidR#835
Basically the idea is to use future to parallel point cloud processing tasks.
It is very useful to perform complete point cloud processing workflow on large datasets and for a large community.
For example, we often use following codes:
The code uses both future and lidR to create a raster of the terrain topography, tile-by-tile and in parallel thanks to future.
At first, the right number of cores is used (n_cores, checked using htop) but with time, this number tends to decrease and reaching the end it is very low (sometimes only 1 core used)
The issue seems to come from future since the lidR code has not changed for a lot of time.
Do you already know about this limitation ?