get_dask_Client

get_dask_Client(timeout=30, n_workers=None, threads_per_worker=1, processes=True, create_cluster=True, memory_limit=None, local_dir='.', with_file=False, scheduler_file='./scheduler.json', dashboard_address=':8787')[source]

Get a Dask.distributed Client for the scheduler defined externally, otherwise create

The environment variable ARL_DASK_SCHEDULER is interpreted as pointing to the scheduler. and a client using that scheduler is returned. Otherwise a client is created

Parameters

  • timeout – Time out for creation

  • n_workers – Number of workers

  • threads_per_worker –

  • processes – Use processes instead of threads

  • create_cluster – Create a LocalCluster

  • memory_limit – Memory limit per worker (bytes e.g. 8e9)

Returns

Dask client