Re: GSoC Free-Lock Hash Table, Multiple library bindings, etc.

[Javier A. G. T. <jav...@gm...>]

Hi Bruno!

> But a Common Lisp binding to libvirt already exists [1], and I would expect
> that it does not take long to make this binding actually run in clisp. (Maybe
> just 1 week?)
>

Yes, but as I can tell, this library is manually made, on the contrary
python libvirt bindings
are automatically created utilizing the API XML descriptions from
libvirt.[1] Part of the project will involve
to mimic the python lib but better (macros and provably run time
actualization as it seems as cool idea but all deepens on the libvirt
C API updates).
1-2 weeks of overhead.

> So, to make this a valuable GSoC project, you would need to add significant
> functionality on top of it. Just "basic functionality of a cloud orchestrator"
> sounds like too little for me.

Ok, a cloud orchestrator composed by a single controller whit N
compute nodes. For communication between nodes I was thinking on
cl-rabbit, where a subset of the C API  is ready. For database, sql
alchemy like library already found con common lisp utilized on
caveman2 web framework project. As you state, multi thread on clisp is
not on production status,
maybe use user-level multi thread library?

Basic functions regarding the orchestrator (utilizing libvirt):
1.- Creation of instances based on flavors (Basic description of
2.- Power operations over instances (Halt, restart, etc.)
3.- Layer 2 Networking including a subset of a firewall (Port-forwarding)
4.- Instance migration

Added functionalities:
1.-Workloads: Description of the OS image, VCPUs, RAM and a bash or
clisp script for pre-configuration.
2.- key-pairs management.
3.- openstack identity service for authentication using its REST API.
4.- OS images management.
5.- Run-time patches. On Open stack this represents a problem to the
point on having a single project to upgrade from a version to
another.[2]


The controller node will perform the  orchestration of the cloud:
1.- Decide where to create a new instance.
2.- Handle the CRUD for networking, key-pairs, OS images and workloads.
3.- Apply patches to other nodes.
4.- Given access only to authenticated nodes and users.

The compute node will handle the operations given by the controller
delivered by the rabbitmq broker and information founded on the
database.

Deliverables:
libvirt
Patches to cl-rabbit and any other library of needed
controller daemon.
compute daemon.

Benefits to CLISP:
1.- More libraries available.
2.- Incentivize cloud development on an already capable language.
3.- A simpler private cloud application where an smaller organization
can deploy their services based on clisp.

Another idea that it comes to my mind is to create a nova service
clone to run on an already deployed open stack cluster to show of
run-time patches.
And a follower to this project, a network node to support layer 3
networking (Networks defined by software).


Background:
1.- sysadmin newbie from both openstack and ciao cluster.
2.- Done basic school work on SBCL. Relevant to this project: a proof
of concept cloud orchestrator using cl-rabbit and cl-vagrant.

Any critique is welcome in order to produce a proposal draft.
Thanks for your time.


PD: Hopefully I can deliver at least the libvirt library on any case.

Instance: Virtual machine already running.
[1] https://libvirt.org/python.html
[2] https://wiki.openstack.org/wiki/Upgrade-with-minimal-downtime