Synchronize from ics file import
Brought to you by:
jberanek
Menu
▾
▴
#320 Synchronize from ics file import
open
nobody
None
1
2015-12-22
2015-12-21
Anonymous
No
Hi,
I've spent a little time investigating how I could synchronise a set of room bookings from an ics file. The idea and situation being that part of our room bookings are managed by an upstream system leaving any remaining slots to be booked by further independant users / system. The upstream system is able to export the core bookingss in iCalendar format and a synchronisation is required to be done remotely either via a cron or a script which is called by the upstream process (to activate the synchronisation).
At the moment I've noticed that I might be able to post a request to the import.php URL hopefully supplying the ical file. However if this (ical) file is being reloaded it is likely that the events already exist in MRBS and only the extra un conflicting bookings will be added, existing books will not be removed or edtited.
What I am really after in the simplest form (via a http or cmmand line interface ) , whcih would attempt to implement some checks before a full synchronisation is:
A method to copy the entries from an exist area into a temporary area.
A method for deleting an entire 'set' of entries (for example identified by the user/worker who uploaded them) - in a given area.
Then the ability to upload an ics file and easily check whether any conflicts existed.
before:
Deleting an entire 'set' of entries (for example identified by the user/worker who uploaded them) in a particular are/room.
Then the ability to upload an ics file and easily check whether any conflicts existed.
This seems to highlight three areas where for my scheme to work I would need to add features:
1, I see a rudimentary way of uploading the ics file however getting any conflict errors might not be as robust as it should be.
2, I see a method to bulk delete entries, though this accepts a series of entry ids, so a assumes the report page has been run first adding an extra step.
3, However I don't see any feature where I am able to bulk copy a set of entries whilst changing some of the entry attributes eg room/area.
How feasible would making improvements to these last three areas be? or indeed is there an alternative scheme that I should be following?
Thanks in advance
Stephen
I think I'm probably missing something, but why can't you just import the .ics file from the upstream process and ignore any conflicts?
If you ignore any conflicts those new/updated events will not be loaded. Any events since removed from the .ics file will also still be in the MRBS.
Are you able to get the upstream process to issue a .ics file every time an event is created, modified or deleted? If so, you could then get MRBS to modify its database as it receives each of those files.
By the way, what stops the upstream process causing an MRBS conflict if one of the upstream events is moved into a previously free slot that has in the meantime been booked by a downstream user?
Yes this is the intention - that the upstream processes issues an updated .ics file which is loaded into the database ( by a manual trigger I hasten to add)
Would you recommend direct access to the database? my preference would be to use the tools built on top of the database persistence, ie any built in procedures that can synchronise the ics file. I was hoping that there were some methods available to acheive this given the support for icalendar uploads.
Nothing will stop someone booking a room outside of the upstream process which would then cause conflict. I was hoping that the method proposed in the OP would catch this and it would then be up to the person causing the upstream process to negotiate the required 'fixes' to the booking
I understood the original plan to be to get the upstream process to issue periodic dumps of its database in .ics format. Then the only way of finding out what has been deleted is to compare a previous dump with the current one.
What I was wondering was whether the upstream process could be configured to send out a .ics file on every change, including deletes (which would have a METHOD:CANCEL)? Then those events could be imported into MRBS one by one.
I agree that it'll be best to use whatever higher level tools MRBS has to offer, although I suspect that whatever one does will involve changing some MRBS code - though this all sounds generally useful.
Yes certainly the upstream process could be developed to export the individual changes that have occurred. However I wouldn't like the two systems to be tied together with this dependency (the upstream app keeping a track of the status of all the downstream apps). I would also suggest that the solution goes beyond just using an data exchange format, and I think you would always need a method to test the integrity of the mirrored dataset no matter how much error checking happens when adding / editing / removing individual events, any way.
If something was available to extract a set of events into ical (based on particular criteria) which could compare this with an incoming ical which would then generate individual actions - that would be fantastic. Though then there would be the issue of feeding back conflicts.
I've had a quick search for PHP libraries which might be able to calculate differences in cal files; though as yet i've not come across any hits.
Wouldn't it be the other way round? In other words the downstream app would keep track of the upstream app? All the upstream app would do is broadcast changes and then it would be up to the downstream app to make the changes and resolve any conflicts?
Yes exactly, but the upstream app would still need to know which changes the downstream app is interested in and have the overhead of a full audit trail to do so. So it might as well broadcast the current state, and the down stream app compare that with its current set to generate the actions.
Ah, OK - you mean that the downstream app is only interested in a subset of the bookings that are handled by the upstream app? So in your scenario, when you export a dump of the upstream app's bookings, you are just exporting a subset, by somehow selecting them for the dump? (Apologies if it's taking me a while to get my head round this!)
Sorry, No the upstream app is dumping ALL its bookings into downstream MRBS. I want to update MRBS with this new picture. Some might have changed, been deleted or been added, or not even changed. I don't see it as sensible that the upstream app supplies individual actions, as to do this it really needs to know hat is currently in the (downstream) MRBS.
cf Google calendars. Imagine if I was was able to supply the URL of my ics file and periodically MRBS downloads and synchronises.
OK, I think I'm beginning to understand.
I think the model I was suggesting was something like somebody inviting me to a meeting. They set up the meeting in their Outlook (or Google or whatever) calendar and invite me as an attendee. I get sent an email notification, with a .ics file, and my calendar app automatically includes that event in my app. When the other person modifies the meeting an update is emailed to me, and my calendar app automatically modifies the meeting in my calendar.
I think the model you are suggesting is similar to subscribing to an external calendar (eg a sports fixture list). When I subscribe to an external calendar from my calendar app, eg Outlook, I think that all that Outlook does is store the calendar data in a read-only folder, and periodically it replaces the contents of that folder with the latest dump from the external calendar.
Both models could work as a solution. In the first model the upstream app broadcasts updates everytime an event has changed. It doesn't need to know what's in the downstream app (MRBS). It just sends all updates to MRBS and MRBS then makes the changes. For example it will send out an update to the event with UID xyz and then MRBS looks for that event and changes it as necessary.
In the second model MRBS periodically imports the contents of the upstream app. When it does the new import it deletes all the old entries and adds the new ones, resolving conflicts with any local events as necessary. You should be able to identify the upstream app events because they should have their own format for the UID, which will be distinguishable from the MRBS style UIDs. So you would do a bulk delete of those and then import the new ones. The easiest way I can see of making all this happen is to modify import.php so that it does the bulk delete (probably with a database level query) and then import the new ones, except for those that conflict with local events. it then emails a list of conflicts to an administrator for resolution. I am assuming in all of this that the imported events are read-only, ie an MRBS user cannot edit an event that has been imported?
So, if I've understood the scenario correctly, I guess what you are looking for is the ability in MRBS to subscribe to an external calendar?
Yes sorry about my roundabout description, and thank you for persevering with me.
I would like to be able to point MRBS to an external calendar and have all the events synchronised into MRBS as read only events; with some sort of conflict management (perhaps the option to rollback if conflicts are found). With the ability to manually trigger when MRBS does the synchronisation.
The upstream process is actually our teaching duties database, which includes the timetable information, so it wouldn't generally change in small increments, but the adminstrators of this tool could change several courses at once over time, depending on enrolment - (just for any case study)
You mentioned the best place to start looking would be in import.php, so perhaps I've concluded there are no higher level tools provided by MRBS which I could utilise. If I were to edit import.php I potentially drift from the main development branch. If I were to attempt to make some changes what sort of likelyhood would there be to get them integrated into the official application? Are there any coding guidelines or formal processes in which enhancements get accepted?
If the changes are generic enough then, yes, there's a good chance of getting them accepted. There's no formal process for acceptance at the moment though. There are also no formal coding guidelines - it's just a case of copy what's there. In due course no doubt we will formalise then and move to something like PSR-2, but we're not there yet. I think the main differences from PSR-2 at the moment are (a) we use indents of 2 spaces not 4, (b) the opening brace for a control structure goes on the next line and not the same line and (c) an else/elseif goes on the next line after a closing curly brace.
You're right - there's nothing higher level than what's there in import.php, though there ought to be, and maybe this would be a good spur to start sorting out some of MRBS. I'd be happy to work with you on this development, tidying up MRBS to provide what you need - for example moving some of the stuff that's currently in import.php out into a class or library.
It sounds like what's needed is something like a subscribe.php which can be run from the command line as a cron job taking a URL as a parameter. It also sounds like the entry table may also need a new field, which is the URL of the calendar you're subscribing to. Although I said that you should be able to distinguish imported events from MRBS events from the format of the UID, this method probably isn't reliable enough for a general solution.
The thing I'm not quite clear about at the moment is how to treat conflicts when you import form the upstream calendar. This problem doesn't occur for calendar apps such as Outlook and google because they allow conflicting events. Maybe MRBS has to import all the events, regardless of whether they conflict, and then flag the conflicts for subsequent resolution.
One other thing that's potentially an issue is the way that MRBS rounds events to slot boundaries. When you import or create an event at the moment it will round the start and times to the nearest slot boundaries. This could cause some problems if the upstream and downstream slot boundaries don't coincide.
Maybe the best thing is to continue this discussion off-list ...