[CAnet - news] A Good Summary of Grid Projects and Developments

For more information on this item please visit my blog at 
http://billstarnaud.blogspot.com/
-------------------------------------------


 [IBM has put together a good list of grid projects around the world -- BSA]

http://www.ibm.com/developerworks/grid/library/gr-gridorgs/index.html

Edna Nerona ([email protected]), Consultant, Consultant

26 Jun 2007

    Previously, we gave you a "Recommended reading list for grid developers"
and "A starter set of open source tools for grid developers." Now we've
compiled a list of some of the production projects and organizations that
are shaping the future of grid computing. This article provides a
comprehensive list of current projects in such diverse areas as cancer
research, astronomy, and physics, just to name a few. We also cover tool
kits, security, and data management. These were taken from a variety of
online sources to introduce programmers, administrators, and new users to
specific information and projects related to using, deploying, and
developing grid infrastructure.

Production grid organizations

This track will introduce you to production grid organizations, the problems
they are solving, and how they're influencing grid technologies. Production
grid deployments fall into various categories of grids: general-purpose
grids, scientific and community grids, nationwide grids, regional grids, and
university grids.

National and international general-purpose grids

Distributed European Infrastructure for Supercomputing Applications
    As a consortium of leading national supercomputing centers, the
Distributed European Infrastructure for Supercomputing Applications (DEISA)
deploys and operates a secure production-quality distributed supercomputing
environment. By enhancing and reinforcing European capabilities in
high-performance computing, the research infrastructure facilitates
scientific discoveries across a variety of science and technology fields.
DEISA uses a deep integration of existing national high-end platforms, with
a dedicated network and support by innovative system and grid software.

DutchGrid
    Established in 2000, DutchGrid has many successful integrated efforts
and initiatives that span a range of scientific collaborations. As an open
platform for academic and research grid computing, DutchGrid provides
globally recognized identity certificates to grid users in the Netherlands.
The DutchGrid CA is fully project-neutral. Any not-for-profit researcher and
academic user can obtain personal and server or host certificates for use
with grid applications.

Enabling Grids for E-science
    The Enabling Grids for E-science (EGEE) project brings together
scientists and engineers from more than 90 institutions in 32 countries
worldwide to provide a seamless grid infrastructure for e-science that is
available to scientists. The EGEE grid consists of more than 30,000 CPUs
available to users 24 hours a day, seven days a week, in addition to about 5
petabytes (5 million gigabytes) of storage, and maintains 30,000 concurrent
jobs on average. Having such resources available changes the way scientific
research takes place. EGEE is a four-year project funded by the European
Commission.

Grid5000
    The purpose of the Grid5000 project is a highly reconfigurable,
controllable, and monitorable experimental grid platform grid researchers
can use as a testbed for experiments in all the software layers between the
network protocols and up to applications. Grid5000 brings together nine
sites geographically distributed in France, featuring 5,000 CPUs. These
areas include Bordeaux, Grenoble, Lille, Lyon, Nancy, Orsay, Rennes,
Sophia-Antipolis, and Toulouse.

LA Grid
    Pronounced "lah grid," the LA Grid is the first-ever comprehensive
computing grid to connect faculty, students, and researchers from
institutions across the United States, Latin America, and Spain to
collaborate on complex industry applications for business and societal needs
in the context of health. In addition to universities, LA Grid has partnered
with industries worldwide, enhancing innovations in many areas, including
healthcare, life sciences and hurricane disaster, life sciences, and
disaster mitigation.

Open Science Grid
    The Open Science Grid (OSG) is a distributed computing infrastructure
for scientific research. The OSG consortium's unique alliance of
universities, national laboratories, scientific collaborations, and software
developers brings petascale computing and storage resources into a shared
uniform cyberinfrastructure.

TeraGrid
    TeraGrid is an open scientific discovery infrastructure funded by the
National Science Foundation. Combining leadership-class resources at nine
partner sites, TeraGrid creates an integrated, persistent computational
resource. Interconnected via a high-speed gigabits-per-second dedicated
national network, TeraGrid provides more than 150 teraflops of computing
power and nearly 2 petabytes of rotating storage, numerous scientific data
collections, specialized data analysis tools, scientific gateways, and user
portals to simplify access to valuable resources, and visualization
resources.

Scientific and community grids

AstroGrid
    AstroGrid is an open source project built to create a working Virtual
Observatory (VO) for U.K. and international astronomers. Funded by the U.K.
government, AstroGrid works closely with other VO projects worldwide through
the International Virtual Observatory Alliance (IVOA). As a leading member
of this community, AstroGrid provides internationally recognized interface
standards that are emerging to promote scientific integration of
astronomical data and processing resources worldwide.

cancer Biomedical Informatics Grid
    The cancer Biomedical Informatics Grid (caBIG) is a voluntary network or
grid connecting individuals and institutions to enable the sharing of data
and tools, creating a worldwide source of cancer research. The goal is to
speed the delivery of innovative approaches for the prevention and treatment
of cancer. The infrastructure and tools created by caBIG also have broad
utility outside the cancer community. caBIG is being developed under the
leadership of the National Cancer Institute's Center for Bioinformatics.

International Virtual Data Grid Laboratory
    The International Virtual Data Grid Laboratory (iVDGL) is a global data
grid that will serve forefront experiments in physics and astronomy. Its
computing, storage, and networking resources in the United States, Europe,
Asia, and South America provide a unique laboratory that will test and
validate grid technologies at international and global scales. Sites in
Europe and the United States will be linked by a multigigabit-per-second
transatlantic link funded by the European DataTAG project.

World Community Grid
    World Community Grid's mission is to create the world's largest public
computing grid to tackle projects that benefit humanity. The success of the
World Community Grid depends upon individuals collectively contributing
their unused computer time to change the world for the better. World
Community Grid is making technology available only to public and
not-for-profit organizations to use in humanitarian research that might
otherwise not be completed due to the high cost of the computer
infrastructure required in the absence of a public grid.

Worldwide Large Hadron Collider Computing Grid
    The Worldwide Large Hadron Collider (LHC) Computing Grid is designed to
handle the unprecedented quantities of data that will be produced by
experiments at CERN's LHC from 2007 onward. The computational requirements
of the experiments that will operate at the LHC are enormous. Some 12-14
petabytes of data will be generated each year, the equivalent of more than
20 million CDs. Analyzing this data will require the equivalent of 70,000 of
today's fastest PCs. The LHC Computing Grid will meet these needs by
deploying a worldwide computational grid, integrating the resources of
scientific computing centers spread across Europe, the United States, and
Asia into a global virtual computing service.


U.S. regional grids

Northwest Indiana Computational Grid
    Northwest Indiana Computational Grid (NWICG) is a partnership of
researchers and educators from Purdue University-West Lafayette, Purdue
University-Calumet, and the University of Notre Dame. With a focus on
national science and research initiatives, NWICG creates cyberinfrastructure
that supports the solution of breakthrough-level problems and enabling
continuing world-class advances in the underlying technologies of
high-performance computing. They are developing a scalable, high-speed,
high-bandwidth, science-driven computational grid for Northwest Indiana
across the three universities in collaboration with the Department of
Energy's Argonne National Laboratories.

SURAGrid
    Southeastern Universities Research Association (SURA) is a consortium of
organizations collaborating and combining resources to help bring grid
technology to the level of seamless, shared infrastructure. The SURAgrid
focuses on direct access to a rich set of distributed capabilities for
participating research and education communities. SURAgrid promotes the
development of contributed resources, project-specific tools and
environments, highly specialized access, and gateways to national and
international cyberinfrastructure.

Texas Internet Grid for Research and Education
    The mission of the Texas Internet Grid for Research and Education
(TIGRE) project is to create a computational grid that brings together
computing systems, storage systems, and databases, visualization
laboratories and displays, and even instruments and sensors across Texas. By
enhancing the computational capabilities for Texas researchers in academia,
government, and industry by integrating massive computing power, TIGRE hopes
to aid in the advancement of biomedicine, energy and the environment,
aerospace, materials science, agriculture, and information technology.


Open source grid projects

These grid projects cover a diverse set of areas, ranging from grid
infrastructure toolkits, middleware toolkits, data tools, security, and
more. The following represent some fast moving projection grid projects and
tools. Visit these sites often to keep up to date on how they are leading
the progress in grid technology.

Grid infrastructure projects

Open source grid infrastructure projects that can help you set up your own
grid.

Globus Toolkit
    Open source software developed by the Globus Alliance. The Globus
Alliance is an international collaboration that conducts R&D to create
fundamental grid technologies. The toolkit includes software for security,
information infrastructure, resource management, data management,
communication, fault detection, and portability. 

Berkeley Open Infrastructure for Network Computing
    Berkeley Open Infrastructure for Network Computing (BOINC) is a software
platform for projects, like distributed.net and SETI@home, that use millions
of volunteer computers as a parallel supercomputer. Source code is available
for the platform, and interested C++ developers are encouraged to help
develop the platform code. BOINC is currently supported on WindowsR, LinuxR,
UNIXR, and Mac OS X. CPU platform requirements may vary among project
clients using BOINC.

Uniform Interface to Computing Resources
    Uniform Interface to Computing Resources (UNICORE) offers a ready-to-run
grid system, including client and server software. UNICORE makes distributed
computing and data resources available in a seamless and secure way in
intranets and the Internet. The UNICORE design focuses several core
principles: seamless access to heterogeneous environments, security, site
autonomy, a powerful GUI clients that provides ease of use, and quick start
bundles that allow for simple installation. 

Grid middleware projects

The following projects have successfully provided U.S. and international
projects with the advanced tools to easily access numerous grid
functionalities, such as computation, visualization, and storage resources.
You can interact with various grids or have one customized to work with your
own grid.

gLite
    gLite is the next generation middleware for grid computing, born from
the collaborative efforts of more than 80 people in 12 academic and
industrial research centers as part of the EGEE Project. gLite provides a
bleeding-edge best-of-breed framework for building grid applications tapping
into the power of distributed computing and storage resources across the
Internet.

National Research Grid Initiative
    The National Research Grid Initiative (NAREGI), in Japan, focuses on the
research and development of grid middleware so that a large-scale computing
environment can be implemented for widely distributed, advanced research and
education.

Ninf-G
    Ninf is a Japanese project developing programming middleware which
enables users to access various resources, such as hardware, software, and
scientific data on the grid with an easy-to-use interface. Ninf-G is open
source software that supports development and execution of grid-enabled
applications using Grid Remote Procedure Call (GridRPC) on distributed
computing resources.

NorduGrid
    NorduGrid middleware, also known as Advanced Resource Connector (ARC),
is an open source software solution distributed under the GPL license,
enabling production-quality computational and data grids. ARC provides a
reliable implementation of the fundamental grid services, such as
information services, resource discovery and monitoring, job submission and
management, brokering and data management, and resource management. Most of
these services are provided through the security layer of the GSI. The
middleware builds upon standard open source solutions like OpenLDAP,
OpenSSL, SASL and Globus Toolkit (GT) libraries. 

OGSA-DAI
    The OGSA-DAI project focuses on the development of middleware to assist
with the access and integration of data from separate sources through the
grid. The project works closely with the Globus, OMII-Europe, NextGRID,
SIMDAT, and BEinGRID, ensuring that the OGSA-DAI software works in a variety
of grid environments.

ProActive
    ProActive is the JavaT grid middleware library (with open source code
under LGPL license) for parallel, distributed, and multithreaded computing.
With a reduced set of simple primitives, ProActive provides a comprehensive
API to simplify the programming of grid computing applications, distributed
on LAN, on clusters of workstations, or on Internet grids.

Security projects

To protect the vital infrastructure and information, security is a constant
evolving requirement of grid computing. These projects represent some of the
cutting-edge security standards and implementations of grid security
solutions.

GridShib
    An NSF-funded project between NCSA and the University of Chicago,
GridShib integrates federated authorization infrastructure (Shibboleth) with
grid technology (the Globus Toolkit) to provide attribute-based
authorization for distributed scientific communities.

Grid User Management System
    The Grid User Management System (GUMS) is a Grid Identity Mapping
Service. Identity mapping is necessary when a site's resources do not use
grid credentials natively, but instead use a different mechanism to identify
users, such as UNIX accounts or Kerberos principals.

PRIvilege Management and Authorization
    PRIvilege Management and Authorization (PRIMA) is a system that provides
enhanced grid security. PRIMA is a comprehensive grid security model and
system. In PRIMA, a privilege is a platform-independent, self-contained
representation of a fine-grain right. PRIMA achieves platform independence
of privileges by externalizing fine-grain access rights to resource objects
from the resource's internal representation.

Resource management and scheduling

An essential component of grids is to manage and schedule jobs across
resources. These projects demonstrate a few strategies.

Community Scheduler Framework
    Community Scheduler Framework (CSF) is an open source implementation of
an OGSA-based metascheduler. It supports the emerging WS-Agreement
specification and the Globus Toolkit's GRAM service. CSF fills in gaps in
the existing resource management picture and is integrated with Platform LSF
and Platform Multicluster. The CSF open source project is included in the
Globus Toolkit V4.0 release.

Special Priority and Urgent Computing Environment
    High-performance modeling and simulation are playing a driving role in
decision-making and prediction. For time-critical emergency support
applications, such as severe weather prediction, flood modeling, and
influenza modeling, late results can be useless. A specialized
infrastructure is needed to provide computing resources quickly,
automatically, and reliably. Special Priority and Urgent Computing
Environment (SPRUCE) is a system to support urgent or event-driven computing
on traditional supercomputers and distributed grids.

Grid resource monitoring

Monitoring resources and applications is key to the success of grids.
Through an easy-to-use interface, these sophisticated tools help users
gather, catalog, and monitor various types of resources. Moreover, systems
administrators are also able to monitor the health of their grids. These
evolving grid projects list a few of the open source options.

GridCat
    GridCat is a high-level grid cataloging system using status dots on
geographic maps, as well as a catalog. The maps help debug site troubles.
The catalog contains information on site readiness with many other valuable
information per site to help job submission and job scheduling for
application users and grid scheduler developers. GridCat tries to present
the grid-site at its simplest status representation.

Gridscape II
    Gridscape II is a customized portal component that can be used on its
own or plugged in to compliment existing grid portals. Gridscape II manages
the gathering of information from arbitrary heterogeneous and distributed
sources and presents them together seamlessly within a single interface. It
leverages the Google Maps API in order to provide a highly interactive user
interface. Gridscape II is simple and easy to use, providing a solution to
those who don't wish to invest heavily in developing their own monitoring
portals from scratch, and also for those who want something easy to
customize.

Storage and data management

>From open source high-performance file systems to seamless access of data
from heterogeneous environments, the following projects bring together and
optimize a variety of storage and data management solutions. This track
emphasizes storing, managing, and moving data across resources and
connecting data resources over a network.

Lustre
    The Lustre File System, a high-performance open source file system from
Cluster File Systems Inc., is a distributed file system that eliminates the
performance, availability, and scalability problems present in many
traditional distributed file systems. Lustre is a highly modular
next-generation storage architecture that combines established open
standards, the Linux operating system, and innovative protocols into a
reliable, network-neutral data storage and retrieval solution. Providing
high I/O throughput in clusters and shared-data environments, Lustre also
provides independence from the location of data on the physical storage,
protection from single points of failure, and fast recovery from cluster
reconfiguration and server or network outages.

NeST
    NeST is a software network storage device providing secured storage
allocation for a specific time period. The size and duration of allocation
units or lots are negotiated between NeST and the user or application. These
lots can also be expanded in size, extended in time and/or subdivided into a
hierarchy. Plus, NeST offers access control lists for lot and file access.
NeST offers multiple protocol interfaces, including its internal Chirp, HTTP
and GSI-FTP. 

SAMGrid
    SAMGrid is a general data handling system designed to be a key device
for experiments with large (petabyte-size) data sets and widely distributed
production and analysis facilities. The components now in production provide
a versatile set of services for data transfer, data storage, and process
bookkeeping on distributed systems.

UberFTP
    Building upon the technologies of GridFTP, UberFTP is the first
interactive GridFTP-enabled FTP client. The basic GridFTP client is not
interactive and allows only one file transfer at a time. UberFTP provides
interactive tools that work much like the popular NCFTP tool. It supports
GSI authentication, parallel data channels, and third-party transfers.



	Back to top


Conclusion

Grid computing is one of the most exciting technologies that are having
powerful effects on the way we solve complex problems and share diverse
resources. In addition to cancer and physics, it also has great influence on
security and authentication, discovery, monitoring, information services,
data management, resource management, and scheduling.


-------------------------------------
To SUBSCRIBE:
send a blank e-mail message to
[email protected]

To UNSUBSCRIBE:
send a blank email message to
[email protected]
-------------------------------------

These news items and comments are mine alone and do not necessarily reflect
those  of the CANARIE board or management.

-----------
[email protected]
[email protected]
www.canarie.ca/~bstarn
skype: pocketpro
SkypeIn: +1 614 441-9603


_______________________________________________
news mailing list
[email protected]
http://lists.canarie.ca/mailman/listinfo/news