Search Results for "compute cluster"

aws-parallelcluster-node is the python package installed on the Amazon EC2 instances launched as part of AWS ParallelCluster. AWS ParallelCluster is an AWS-supported Open Source cluster management tool that makes it easy for you to deploy and manage High-Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. ...

omega|ml is the innovative Python-native MLOps platform that provides a scalable development and runtime environment for your Data Products. Works from laptop to cloud.

Jupyter Enterprise Gateway is a headless web server with a pluggable framework for anyone supporting multiple notebook users in a managed-cluster environment. Some of the core functionality it provides is better optimization of compute resources, improved multi-user support, and more granular security for your Jupyter notebook environment - making it suitable for enterprise, scientific, and academic implementations. From a technical perspective, Jupyter Enterprise Gateway is a web server that enables the ability to launch kernels on behalf of remote notebooks. ...

dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing.

Manage node sets, node groups and execute commands on cluster nodes in parallel. Provides an event-based Python library to improve administration of large compute clusters or server farms. Command line tools: clush and nodeset included.

Cluster Bench is a framework to support a range of benchmarks and system tests for cluster systems. It automatically generates the required files for a batch system and can graphically display the results of completed benchmarks and other tests.

RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. ...

...Functions include: duplicate removal demultiplexing primer-removal quality-assurance (QA) graphing quality control (QC) consensus-generation minority-variant determination minority-variant graphing The main current version is 6.X, which is written in Python3. 7.X is my rewrite in Java, but is still work in progress. Both are written to be as lightweight as possible so they can run with minimal memory-requirements on a desktop or laptop as well as on a compute cluster. If you have any problems with QUASR, please do contact me at the email address provided in the README.