============== Z-Push Zimbra Backend - Release Notes - Release 72 ==============
THESE RELEASE NOTES ARE FOR RELEASE 72 CHANGES
** RELEASE 72 is designed to work with Z-Push 2.5 or later **
PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 72
NOTE THAT THE RELEASE NOTES FOR 37-71 ARE ALSO CONTAINED BELOW AND SHOULD
ALSO BE READ FULLY WHEN UPGRADING FROM A RELEASE 48 OR EARLIER.
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES
AND ARE UPGRADING AN EXISTING INSTALLATION FROM RELEASE 48 OR EARLIER.
SHOW YOUR APPRECIATION
If you would like to show your appreciation for the work that goes into this zimbra backend
please send a donation via
https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=7X94XCP9A5ZYG&lc=IE¤cy_code=EUR&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted
New for Release 72 and later. Three new config settings have been added.
ZIMBRA_SYNC_WINDOW_DAYS allows overriding the default sync max window of +/- 366 days
ZIMBRA_SYNC_GAL_GROUPS allows overriding the default of not syncing GAL groups
ZIMBRA_USE_MB_CHECK_ENCODING allows for PHP 8.1+ to use traditional preference ordered
searching for source encoding. See details of each below.
New For Release 71 and later. A new config file flag ZIMBRA_DELETESASMOVES was added that
allows the default behaviour of zimbra (always send deleted stuff to trash) to be overridden
such that the flag sent by the client will be honoured instead. Please manually add this new
setting to your config.php file if you wish to use it. Also, as the zimbra backend local
copy of z_RTF.php is no longer used, the latest version of zpzb-install.sh script will rename
it. So, please download and use the latest script.
NEW For Release 63 and later. The code to recognise the UserAgent of various devices and to
enable multi-folder support where appropriate has been reworked. The configuration information
is now broken out into a separate file zimbraMultiFolderUAs.php. To add multi-folder support
for any new device simply edit this file and add a new line to the array, setting the appropriate
switches for the different folder types. An upcoming change to the z-push core will allow
retrieval from the state files of the UserAgent for clients such as some Windows clients that do
not supply their UserAgent on every request.
STARTING WITH Release 57 - SmartFolders ARE ENABLED BY DEFAULT
If you are upgrading and wish to continue to use XML files, you need to ensure that you have
define('ZIMBRA_SMART_FOLDERS',false);
in your zimbra backend config.php file.
ZIMBRA 8.0 AND LATER - DoSFilter MUST whitelist z-push server
Assuming that z-push is not running on the same IP address as zimbra, if you are running
zimbra 8.0 or later, you MUST whitelist the IP Address of the z-push server to avoid
connection issues. See http://wiki.zimbra.com/wiki/DoSFilter
For 8.0.0-8.0.2 see
http://www.zimbra.com/forums/announcements/60397-zcs-dosfilter-workaround-zcs-8-0-1-8-0-2-a.html
ZIMBRA 7.2 AND LATER - SOAP Session limit warning
If you have users with multiple devices synching against accounts, be aware that there
is a potential issue with the number of allowed SOAP sessions per user. This was
highlighted in a Zimbra Bugzilla ticket and was fixed in zimbra release 7.2 and later,
where you can now increase the number. See https://bugzilla.zimbra.com/show_bug.cgi?id=75338
for details. I set the value to 7 on my server and typically don't see the issue any more.
Symptoms of this problem include the reading pane of the Web Client suddenly clearing while
you are reading email, and loops in activesync where the session disappears in the middle
of processing.
ZIMBRA 6.0.4 AND EARLIER - BROKEN WITH Z-Push 2
These releases of zimbra have a bug in the JSON output of commands that breaks the
json_decode parser. If you wish to use z-push-2 with the zimbra backend you must update
to a newer zimbra release. See zimbra bugzilla for details :-
http://bugzilla.zimbra.com/show_bug.cgi?id=42867
Z-PUSH 2.0.x needs changes to 4 function definitions
If you plan to use the backend with Z-Push release 2.0.x - you really ought to upgrade to
Z-Push 2.4.x or later. There are many imporvements in functionality and device support.
If you do need to stick with the 2.0.x branch there are four function definitions in
zimbra.php that will need to be updated. Each one will need to have a default value added
to the final (contentParameters) parameter. The function headers should look like this :-
public function ChangeMessage($folderid, $id, $input, $contentParameters=false) {
public function DeleteMessage($folderid, $id, $contentParameters=false) {
public function MoveMessage($folderid, $id, $newfolderid, $contentParameters=false) {
public function SetReadFlag($folderid, $id, $flags, $contentParameters=false) {
Use a text editor to search for each of the functions in turn, and add the =false as shown.
Changes Made To Revision 72: z-push-2 version ONLY
- GetNextMessageBlock - Moved optional parameter output to last place
- Changed range handling in GetGALSearchResults
- Added config.php option ZIMBRA_SYNC_WINDOW_DAYS
- Add Auth Failed/Token Expired handling for XML responses
- New setting ZIMBRA_SYNC_GAL_GROUPS
- Ensure $bodyPrefArray is always an array, never false, for PHP 8.1+
- New setting ZIMBRA_USE_MB_CHECK_ENCODING for PHP 8.1+
- Add handling for Zimbra 10 "Files shared with me"
GetNextMessageBlock - Moved optional parameter output to last place
===================================================================
GetNextMessageBlock - Moved optional parameter output to last place in the list so as to
be compatible with PHP 8.x
Changed range handling in GetGALSearchResults
=============================================
Changed range handling in GetGALSearchResults as iOS devices running recent iOS versions
are passing an unexpected search range in the request. The change will not limit the
search range accepted
Added config.php option ZIMBRA_SYNC_WINDOW_DAYS
===============================================
By default the zimbra client uses a 366 day before and after window for choosing which
Calendar/Task/Note items to sync to the device. This arbitrary limit was selected when
the devices connecting were showing a limited window of data. However, now that people
are connecting with Outlook and expecting everything to be synced this is no longer
ideal. The setting ZIMBRA_SYNC_WINDOW_DAYS can override that default number. In order
to specify a different value (from the default of 366) uncomment this setting and set
it to the appropriate value. Change this setting at your own risk as widening the
window could have unexpected consequences. Regardless of what value is set here, if
the client supplies a cutoffdate for searching backwards that value will be used.
// define('ZIMBRA_SYNC_WINDOW_DAYS', 366);
Add Auth Failed/Token Expired handling for XML responses
========================================================
Add Auth Failed/Token Expired handling for XML responses in function SoapRequest to
match the handling for JSON responses - throw new AuthenticationRequiredException
New setting ZIMBRA_SYNC_GAL_GROUPS
==================================
Historically, the Zmbra Backend has not synced any "group" contacts to the devices.
Feature Request #30 - https://sourceforge.net/p/zimbrabackend/feature-requests/30
has argued the case for allowing GAL groups to sync while still blocking user created
groups as before. Adding a new setting ZIMBRA_SYNC_GAL_GROUPS to enable syncing GAL
groups for those that wish to do so. The default value will be false to maintain
the status quo for those who choose not to set a value
// define('ZIMBRA_SYNC_GAL_GROUPS', false);
Ensure $bodyPrefArray is always an array, never false, for PHP 8.1+
===================================================================
Ensure $bodyPrefArray is always an array, never false, for PHP 8.1+ as it throws
an error
New setting ZIMBRA_USE_MB_CHECK_ENCODING for PHP 8.1+
=====================================================
PHP 8.1 changed the way mb_detect_encoding works (see details of the change in
https://github.com/php/php-src/issues/8279). For some users this has caused encoding
problems. As outlined in 4th to last comment on the issue, a loop of calls to
mb_check_encoding can emulate the previous behaviour. For those that need the
previous behaviour in the backend a new setting has been added. The default value
of false will continue to use the mb_detect_encoding but with the new detection
methodology. Setting the value to true will loop through the encodings specified
(or defaulted) in ZIMBRA_MB_DETECT_ORDER returning the first one that passes
mb_check_encoding
// define('ZIMBRA_USE_MB_CHECK_ENCODING', false);
Add handling for Zimbra 10 "Files shared with me"
=================================================
Add handling for Zimbra 10 "Files shared with me". This is an experimental fix for now
as I am currently unable to figure out how to populate the folder - perhaps it is a
NETWORK edition feature related to the zimbra drive?? Fix for now is to exclude the
folder from syncing
Changes Made To Revision 71: z-push-2 version ONLY
- Removed zimbra specific version of include file z_RTF.php
- Don't output "" for empty Task dates and reminders
- Changed mail.NO_SUCH_WAITSET to LOGLEVEL_DEBUG
- Replaced all references to debugLog with ZLog::Write(LOGLEVEL_DEBUG,
- New override setting ZIMBRA_DELETESASMOVES added
- Mail_Mime files mime.php and mimepart.php updated to release 1.10.10
- Don't fail recurrence comparison on firstdayofweek value
- Don't fail recurrence comparison on until value if less than a day differece
- Don't fail recurrence comparison on until value if not present in input
- Removed use of each() in timezone matching for PHP 8 compatability
Removed zimbra specific version of include file z_RTF.php
=========================================================
Removed zimbra specific version of include file z_RTF.php that is now included in the
base z-push packages. The z-push version has been enhanced.
Don't output "" for empty Task dates and reminderset
====================================================
Don't output "" for empty Task dates and reminderset. The empty space values were causing
the z-push integrity checker to throw up errors. Changed reminderset to return 0 and
removed setting of default empty string values for date completed, startdate, utcstartdate,
duedate, utcduedate
Changed reporting of error mail.NO_SUCH_WAITSET to LOGLEVEL_DEBUG
=================================================================
Changed reporting of error mail.NO_SUCH_WAITSET to LOGLEVEL_DEBUG as it can occur normally
when syncing. This will prevent it flooding the z-push-error.log file. It will still show
in the z-push.log and special log files at LOGLEVEL_DEBUG or higher
Replaced all references to debugLog with ZLog::Write(LOGLEVEL_DEBUG,
====================================================================
Replaced all references to debugLog with ZLog::Write(LOGLEVEL_DEBUG, as the old function
no longer exists
New override setting ZIMBRA_DELETESASMOVES added
================================================
New override setting ZIMBRA_DELETESASMOVES added to allow the administrator to force
bypassing the Trash folder for Deletions. By default the zimbra client uses the Trash
folder for deletions of mail messages, tasks and contacts. This has always been the same
in the ZimbraBackend for consistency. For those that would prefer to control the use of
the Trash folder from their client a new setting ZIMBRA_DELETESASMOVES has been added
that can be used to override the default behaviour. The default for this setting is true
which means it will move items to the Trash folder when they are deleted on the client.
To bypass the Trash folder and delete them immediately simply uncomment the following
line in the zimbra backend config.php file and set it to false. BEWARE that it is a
system-wide setting that will affect ALL users.
define('ZIMBRA_DELETESASMOVES', false);
Mail_Mime files mime.php and mimepart.php updated to release 1.10.10
====================================================================
Mail_Mime files mime.php and mimepart.php updated to release 1.10.10 for bug fixes and
to remove old debugLog calls. No changes to ZimbraBackend use of them made at this time.
Don't fail recurrence comparison on firstdayofweek value
========================================================
Don't fail recurrence comparison on firstdayofweek value as zimbra typically does not
store it or return it for recurrences. the zimbra backend does not use this value at
present so set the zimbra firstdayofweek value to match the input one so the
comparison does not break. After the comparison is done the value is dropped - it does
not get stored in zimbra.
Don't fail recurrence comparison on until value if less than a day differece
============================================================================
Don't fail recurrence comparison on until value if less than a day difference as zimbra
typically stores the starttime of the last appointment in the sequence in until whereas
Apple and possibly other clients will send the endtime of the appointment. Assign the
current zimbra until to input until so the comparison does not break.
Don't fail recurrence comparison on until value if not present in input
=======================================================================
Don't fail recurrence comparison on until value if not present in input as a never
ending recurrence created on Apple will have until empty whereas zimbra will have set
an arbitrary end date 10years and a few days out. Assign the current zimbra until to
input until so the comparison does not break.
Removed use of each() in timezone matching for PHP 8 compatibility
==================================================================
Removed use of each() in timezone matching for PHP 8 compatibility. Thanks to Soner
for pointing out the issue https://sourceforge.net/p/zimbrabackend/feature-requests/27/
Note: the z_RTF.php issues listed are no longer relevant due to the removal of the
zimbra backend version of this file - see first change in this release.
Changes Made To Revision 70: z-push-2 version ONLY
- Workaround Z-Push issue in SmartForward/SmartReply from shared folder
- Add 2 to continue statement in GetNextMessageBlock for Contacts
- Throw AuthenticationRequiredException for AUTH failures
Workaround Z-Push issue in SmartForward/SmartReply from shared folder
=====================================================================
Z-Push has logic that splits the itemId on a ":" in SmartForward/SmartReply and only
returns the part after the ":" as the itemId. This is based on the fact that Exchange/
Kopano use folderId:itemId as a long form identifer for an item, and the folderId is
already being passed as a separate field. For zimbra this does not work if the item is
from a shared folder, as the itemId from zimbra will be userGuid:itemId, and when they
split the itemId and only return the latter part, this itemId no longer has any
reference back to the owner. The workaround added is to check if the folderId is for a
shared folder and, if it is, to extract the folder owner's userGuid from the folder
link and to prepend "<userGuid>:" to the itemId. Issue first reported at
https://sourceforge.net/p/zimbrabackend/support-requests/181/
Add 2 to continue statement in GetNextMessageBlock for Contacts
===============================================================
Add 2 to continue statement in GetNextMessageBlock for Contacts to correctly skip the
remainder of the loop, and to avoid generating warnings with PHP 7.3 and later. Issue
first reported at https://sourceforge.net/p/zimbrabackend/bugs/90/
Throw AuthenticationRequiredException for AUTH failures
=======================================================
Throw AuthenticationRequiredException for AUTH failures to allow clients to recognized
the need to re-authenticate. Throwing this exception results in a HTTP 401 response
being returned to the client. Prior to this it was returning a generic Server Error.
Changes Made To Revision 69: z-push-2 version ONLY
- Added descriptive WARN message for unavailable shared folder
- In isZimbraObjectInSyncInterval treat no response as false
- Fix processing of zimbraMailAlias to handle string if exactly one alias
- Fix ChangeMessage to strip input Categories from shared folder items
- Renamed constructor function of mime.php, mimePart.php and z_RTF.php
- Added hash to Primary folder stats to improve virtual folder change detection
- Added logic to clear cache on Logoff where folder changes are detected
- Updated comment and removed extra debug logging from recent fixes
Added descriptive WARN message for unavailable shared folder
============================================================
Added descriptive WARN message for unavailable shared folder to prevent an Unidentified
index warning getting written to the z-push-error.log file.
In isZimbraObjectInSyncInterval treat no response as false
==========================================================
In isZimbraObjectInSyncInterval treat no response as false to prevent trying to operate
on an item that was not found
Fix processing of zimbraMailAlias to handle string if exactly one alias
=======================================================================
Fix processing of zimbraMailAlias to handle string if exactly one alias. Previously the
code assumed that zimbraMailAlias would either not exist (in the case of no aliases) or
would always contain an array of email aliases. In fact, if there is exactly one alias
configured for an account zimbra will return a string object containing that one alias
rather than an array containing one element containing the alias.
Fix ChangeMessage to strip input Categories from shared folder items
====================================================================
Fix ChangeMessage to strip input Categories from shared folder items. While the code
previously prevented modifying the categories(tags) on items from shared folders it did
not strip out the Categories for new items being created in shared folders. If Outlook
for example passed over a Colour category attached to a new item the creation of the item
on zimbra would fail due to the non-existence of the matching tag.
Renamed constructor function of mime.php, mimePart.php and z_RTF.php
====================================================================
Renamed constructor function of mime.php, mimePart.php and z_RTF.php for PHP 7.2/7.3
compatability
Added hash to Primary folder stats to improve virtual folder change detection
=============================================================================
Z-Push calls GetFolderStats() to compare what it knows to what the server currently has
for each folder. In the zimbra backend this is implemented by returning a string
containing the Folder name and metadata. For situations (Outlook Contacts for example)
where virtual folders are needed, a change to an item in any folder other than the
Primary one will get flagged and the stats of it's folder will get updated. But as the
z-push server only knows to query the stats of the Primary so it will never realise it
needs to do a refresh on the virtual folder. We need to add an indicator to the Primary
folder stats to alert about a change on any virtual folder. This has been implemented
by stringing together hashes of the stats from all virtual folders of the primary
folder's type, and adding a hash of the resultant string to the stats returned by the
backend for the Primary folder. Any change to the virtual folders should change the
hash and cause the change in the virtual folder being synced to the device.
Added logic to clear cache on Logoff where folder changes are detected
======================================================================
Added logic to clear cache on Logoff where folder changes are detected. This should
aid in the detection of Hierarchy Changes
Updated comment and removed extra debug logging from recent fixes
=================================================================
Updated comment and removed extra debug logging from recent fixes
Changes Made To Revision 68: z-push-2 version ONLY
- Rename BackendSearchZimbra constructor for PHP 7+ compatability
- Add third parameter to definition of GetGALSearchResults() for Z-Push 2.4
- Added " around $zimbraFolderId for Task/Note in GetNextMessageBlock
Rename BackendSearchZimbra constructor for PHP 7+ compatability
===============================================================
Rename BackendSearchZimbra constructor for PHP 7+ compatability. Class constructors
must now be named __construct(...). The old method of creating a function with the
same name as the class has been deprecated.
Add third parameter to definition of GetGALSearchResults() for Z-Push 2.4
=========================================================================
Add third parameter to definition of GetGALSearchResults() for Z-Push 2.4
compatability. This new parameter relates to the use of pictures in the GAL. It
should be noted that zimbra does not support pictures in the admin tool so the GAL
requests will never return pictures to the client regardless of the values sent.
Added " around $zimbraFolderId for Task/Note in GetNextMessageBlock
===================================================================
Added " around $zimbraFolderId for Task/Note in GetNextMessageBlock as without them
the SOAP parser was failing in the latest (at time of writing) zimbra releases
Changes Made To Revision 67: z-push-2 version ONLY
- Allow for iPxx device meeting modification request with shadow data
- Changed ZIMBRA_DEBUG logging of folder setup to only fire on word 'setup'
- Move GetMailboxSearchResults() Log line to avoid warning on no folder Id
- Tidy up logging in GetMailboxSearchResults()
- Add debug logging of unidentified incoming Timezone in ChangeMessage()
- unset $preModAppt->recurrence->premodtype after using it to fix type
- Log error and return false if GetMsgResponse is not present in response
- Add config.php option ZIMBRA_DISABLE_BIRTHDAY_SYNC
- Don't allow difference in dtstamp of appointment exceptions cause an issue
- Keep original Organizer for Tasks in ChangeMessage
- Check for service.PROXY_ERROR in Login and ChangesSink in multi-server setup
- If no Change Token is returned from CreateWaitSet then delay and return
- Updated comment on FakeOutbox
Allow for iPxx device meeting modification request with shadow data
===================================================================
Allow for iPxx device meeting modification request with shadow data. This change
is required because newer iOS versions do not send complete information for some
items they are changing. Use data from the Pre-Mod object to populate the missing
properties for comparison purposes
Changed ZIMBRA_DEBUG logging of folder setup to only fire on word 'setup'
=========================================================================
Changed ZIMBRA_DEBUG logging of folder setup to only fire on word 'setup' in order
to reduce the amount of logging noise when trying to track down issues elsewhere.
Move GetMailboxSearchResults() Log line to avoid warning on no folder Id
========================================================================
Move GetMailboxSearchResults() Log line to avoid warning on no folder Id. This was
reported on the SourceForge site. The log lines were not intended to remain in the
code beyond the testing phase for the search function
Tidy up logging in GetMailboxSearchResults()
============================================
Tidy up logging in GetMailboxSearchResults() as these were just in place to validate
the options being presented by various mail clients
Add debug logging of unidentified incoming Timezone in ChangeMessage()
======================================================================
Add debug logging of unidentified incoming Timezone in ChangeMessage(). In the case
where an incoming change has an unrecognized timezone name it should be recorded in
the log file with a view to being added to the zimbraNonPhpTimezones.php file with
the correct mapping.
unset $preModAppt->recurrence->premodtype after using it to fix type
====================================================================
unset $preModAppt->recurrence->premodtype after using it to fix type as to leave
this property in place would automatically cause the equals() comparison to fail
Log error and return false if GetMsgResponse is not present in response
=======================================================================
Log error and return false if GetMsgResponse is not present in response. This
condition was observer once in the z-push-error.log but no other logging was
available to track down how it could have happened.
Add config.php option ZIMBRA_DISABLE_BIRTHDAY_SYNC
==================================================
Add config.php option ZIMBRA_DISABLE_BIRTHDAY_SYNC. This has been added as a fail-
safe due to the removal of the limitation on only allowing the birthday field to
sync out to Apple and Nokia devices. Some early Android devices would constantly
cycle on requesting Contacts with a birthday field which would drain their battery
so the original restriction was put into the code. It seems it is no longer needed.
Don't allow difference in dtstamp of appointment exceptions cause an issue
==========================================================================
Don't allow difference in dtstamp of appointment exceptions cause an issue. Even a
small difference (millisecond(s)) in the dtstamp of an exception to a series can
cause a mail.INVITE_OUT_OF_DATE error when trying to modify an exception that did
not need to be touched
Keep original Organizer for Tasks in ChangeMessage
==================================================
Keep original Organizer for Tasks in ChangeMessage as when editing a task in a shared
folder an error will occur if the organizer is not the original one
Check for service.PROXY_ERROR in Login and ChangesSink in multi-server setup
============================================================================
Check for service.PROXY_ERROR in Login and ChangesSink in multi-server setup as if
the proxy is running, and the MTA is running but the user's mailbox server is down
they will receive an AuthToken from the proxy but subsequent requests will fail
due to the fact that they have to be handled by the user's mailbox server
If no Change Token is returned from CreateWaitSet then delay and return
=======================================================================
If no Change Token is returned from CreateWaitSet then delay and return just as we
do if there is a SOAP Error. This condition can occur in multi-server environments
when an individual's Mailbox server goes offline
Updated comment on FakeOutbox
=============================
Updated comment on FakeOutbox to remove references tieing the issue to just Nokia and
Android devices
Changes Made To Revision 66: z-push-2 version ONLY
- Use zimbraHttpStreamWrapper class to output MIME body
- Log ERROR if setting "zimbraAttachmentsBlocked" is "TRUE"
- Fix passing header on GetFolder for shared folders generates new session
- Replace split() with explode() for PHP 7 compatability
- Refactored GetMessageList() to improve efficiency and reduce memory needed
- Added config.php options ZIMBRA_SSL_VERIFYPEER and ZIMBRA_SSL_VERIFYHOST
- Added SSL Verify Peer and Host options to zimbraHttpStreamWrapper class
Use zimbraHttpStreamWrapper class to output MIME body
=====================================================
Use zimbraHttpStreamWrapper class to output MIME body to ensure that the client will receive
a content length
Log ERROR if setting "zimbraAttachmentsBlocked" is "TRUE"
=========================================================
Log ERROR if setting "zimbraAttachmentsBlocked" is "TRUE". This has come from a troubleshooting
a ticket where the mobile client was having issues with attachments. The root cause ended up
being this zimbra setting preventing the user from handling attachments.
Fix passing header on GetFolder for shared folders generates new session
========================================================================
Fix passing header on GetFolder for shared folders generates new session. A change introduced in
Release 64 to get more information on shared folders added an additional call to GetFolderInfo.
This call mistakenly had the header argument passed to it which resulted in the backend
generating a new zimbra session leaving the original one in the background. Logoff is not able
to close this extra session. Ultimately this resulted in lots of "Too many SOAP sessions" errors
in the mailbox.log file.
Replace split() with explode() for PHP 7 compatability
======================================================
Replace split() with explode() for PHP 7 compatability. split() was deprecated in PHP 5.3 and
removed in PHP 7. See http://php.net/manual/en/function.split.php and thanks to user icebreacker00
for reporting the issue - https://sourceforge.net/p/zimbrabackend/bugs/64/
Refactored GetMessageList() to improve efficiency and reduce memory needed
==========================================================================
Refactored GetMessageList() to improve efficiency and reduce memory needed. The message list is
now passed by reference and the amount of recursion needed has been greatly reduced. The code
has also been broken down into several reusable functions to cut down on code duplication.
Added config.php options ZIMBRA_SSL_VERIFYPEER and ZIMBRA_SSL_VERIFYHOST
========================================================================
Added config.php options ZIMBRA_SSL_VERIFYPEER and ZIMBRA_SSL_VERIFYHOST for compatability with
php 5.6 and later SSL connections to a zimbra server configured with a non-commercial certificate.
The default behaviour in PHP 5.6 and later has been changed to verify host names and certificates.
This has never been required before. Two flags are available to configure the SSL context options
for both cURL and Streams as appropriate for each installation. If the settings are not used or
are defined and set to false the behaviour will be as before - no validation of the host or it's
name will be performed. Setting either flag to true will result in that check being perfomed for
each cURL or Stream session opened.
Added SSL Verify Peer and Host options to zimbraHttpStreamWrapper class
=======================================================================
Added SSL Verify Peer and Host options to zimbraHttpStreamWrapper class in order to facilitate
the opening of streams using SSL with PHP 5.6 and later
Changes Made To Revision 65: z-push-2 version ONLY
- Added zimbraHttpStreamWrapper class to provide attachment length to streamer
- If inv[0] of a Calendar item is an Exception then don't treat it as one
- Fixed issue with timezone identification where DST is not observed
- Fixed issue with population of Timezone object in function GetTz()
- Added handling of Company Main Phone field and nickName
- Added readonly parameter to Setup() for z-push 2.3 support
- Updated GetInvIDFromMsgID to retrieve InvID from Message for exceptions
- Commented out some debug logging
- Incorrect variable name used in unlink command - Check for orphaned files
Added zimbraHttpStreamWrapper class to provide attachment length to streamer
============================================================================
Added zimbraHttpStreamWrapper class to provide attachment length to streamer as
it is required by the streamer, and was not being provided by the zimbra http
stream directly. This should fix an issue with images on BB and WP devices and
forwarding emails with attachments on iOS
If inv[0] of a Calendar item is an Exception then don't treat it as one
=======================================================================
If inv[0] of a Calendar item is an Exception then don't treat it as one. This
better handles the case for an exception to an appointment series where an edit
has been made to just one occurrence
Fixed issue with timezone identification where DST is not observed
==================================================================
Fixed issue with timezone identification where DST is not observed. The standard
php timezone libraries do not always provide the up-to-date timezone definitions
first. This change seeks to make more reliable matches where a zimezone identifier
is not passed from the device
Fixed issue with population of Timezone object in function GetTz()
==================================================================
Fixed issue with population of Timezone object in function GetTz(). The stdbias and
dstbias settings were not always being computed correctly
Added handling of Company Main Phone field and nickName
=======================================================
Added handling of Company Main Phone field and nickName as fields within a contact.
Even though "nickName" is not offered as an option within the zimbra web client, if
a custom field is added, and the name set to nickName (case sensitive) it should sync
Added readonly parameter to Setup() for z-push 2.3 support
==========================================================
Added readonly parameter to Setup() for z-push 2.3 support. The setup function
definition was changed by the z-push team for version 2.3. This broke the zimbra
and all other custom backends. As the read-only parameter is irrelevant on zimbra
just adding a defaulted placeholder parameter to the backend fixes the issue
Updated GetInvIDFromMsgID to retrieve InvID from Message for exceptions
=======================================================================
Updated GetInvIDFromMsgID to retrieve InvID from Message for exceptions as these are
handled differently under the covers by zimbra. Hopefully this will now work in mostsituations
Commented out some debug logging
================================
Commented out some debug logging in order to clean up the logs
Incorrect variable name used in unlink command - Check for orphaned files
===========================================================================
Incorrect variable name used in unlink command in function UploadToZimbra(). This error was
introduced in Release62 with the cleanup of some variable names in the code. The function is
used for uploading contact pictures changed on devices. As a result of the error the temporary
files generated for any device-updated pictures since the installation of Release62 will not
have been deleted from the folder. Please check for and delete any orphaned files that have
been generated. The code places the files in the system temporary storage area, and the files
will be named ZIMBRA_FILE_?????.png'
Changes Made To Revision 64: z-push-2 version ONLY
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Correct path to backend multi-folder support file for autodiscover
- Identify character encoding of Attachment name and re-use when forwarding
- Remove closing PHP tag from files
- Remove unused experimental function CustomRowCmp
- Set X-Forwarded-For header to Request::$RemoteAddr if available
- Added _ua string 'MSFT-WP/10' to the zimbraMultiFolderUAs.php file
- Save folder permissions in folder array/cache
- If shared calendar with write permision do not output Organizer
- Added _ua string 'Android/6' to the zimbraMultiFolderUAs.php file
- Add try/catch around call to ZPush::GetDeviceManager()->GetUserAgent()
- Support new names for Z-Push official ResolveRecipients classes
- Added code to handle SyncBaseBody as a stream for Z-Push 2.3 and later
- For deleted recurring meetings set meetingstatus to null - don't unset it
- For recurring meetings do not output meetingstatus on exceptions in AS 2.5
- Updated Out-Of-Office to handle different External messages
- Handle both Autodiscover and Browser Tests when setting the _ua string
- Allow a default user XML file to be used for all users
- Add support for category changing to messages and to _cachedMessageLists
- Renamed constructor function from BackendZimbra to __construct
- Removed //IGNORE//TRANSLIT from $params array in SendMail
- Added new config setting 'ZIMBRA_URL_ALLOW_REDIRECT' for OPEN LDAP auth
- Added SmartFolders flag to cache to detect changes and invalidate cache
- Added stats field to _folders for use by new FolderStats functionality
- Added extra GetFolder calls to get data on Shared Folders for FolderStats
Correct path to backend multi-folder support file for autodiscover
==================================================================
Correct path to backend multi-folder support file for autodiscover, as the path for the
include was from the z-push base directory. This is different from the Autodiscover base
directory. Removing 'autodiscover/' from the base path allows the backend folder to be
referenced correctly. This was a purely cosmetic error which did not prevent autodiscover
from working.
Identify character encoding of Attachment name and re-use when forwarding
=========================================================================
Identify character encoding of Attachment name and re-use when forwarding. The code to add
attachments back to Smart-Forwarded emails was not supplying the character encoding of the
filename or the header encoding to use. In the absense of providing these the code defaulted
to ASCII which is fine for most countries but a Russian user reported that the filenames were
becoming unreadable for some receiving clients while other clients had no issues. Code has
been added to try to identify the character set of the file name text, and to save it in the
attachment object for re-use in the SemdMail code, and base64 encoding have been specified
for the filename fields in the headers.
Remove closing PHP tag from files
=================================
Remove closing PHP tag from files according to latest best practice to avoid outputting
blank lines to output data
Remove unused experimental function CustomRowCmp
================================================
Remove unused experimental function CustomRowCmp as from z-push 2.2.4 onwards the original
DiffState function RowCmp that caused problems for zimbra users has been removed from z-push.
A new more efficient state comparison method has been introduced that eliminates the need for
the problematic sorting of arrays
Set X-Forwarded-For header to Request::$RemoteAddr if available
===============================================================
Set X-Forwarded-For header to Request::$RemoteAddr if available so that in instances where
Z-Push itself is behind a reverse proxy the device IP will be passed on to zimbra instead of
proxy IP. Z-Push already checks for an x-forwarded-for header if the setting is defined in
the z-push config.php - define('USE_X_FORWARDED_FOR_HEADER', true);
Added _ua string 'MSFT-WP/10' to the zimbraMultiFolderUAs.php file
==================================================================
Added _ua string 'MSFT-WP/10' to the zimbraMultiFolderUAs.php file. Identification of Windows
Mail clients requires z-push 2.2.4 or later.
Save folder permissions in folder array/cache
=============================================
Save folder permissions in folder array/cache for use in deciding if we have write permissions
on shared folders
If shared calendar with write permision do not output Organizer
===============================================================
If shared calendar with write permision do not output Organizer as doing so will prevent the
user from being able to edit the appointment/meeting on some devices.
Added _ua string 'Android/6' to the zimbraMultiFolderUAs.php file
=================================================================
Added _ua string 'Android/6' to the zimbraMultiFolderUAs.php file.
Add try/catch around call to ZPush::GetDeviceManager()->GetUserAgent()
======================================================================
Add try/catch around call to ZPush::GetDeviceManager()->GetUserAgent(). This prevents a
fatal error when the Z-Push test page is called up from a web browser.
Support new names for Z-Push official ResolveRecipients classes
===============================================================
Support new names for Z-Push official ResolveRecipients classes. It has taken almost two
years to get z-push to accept this contributed code, and when they finally did they changed
some of the class names. The backend will handle both the original and new class names.
Added code to handle SyncBaseBody as a stream for Z-Push 2.3 and later
======================================================================
Added code to handle SyncBaseBody as a stream for Z-Push 2.3 and later. Code should also
handle older z-push versions where the data is expected to be a text string
For deleted recurring meetings set meetingstatus to null - don't unset it
=========================================================================
For deleted recurring meetings set meetingstatus to null - don't unset it. Unsetting it
was causing a warning in the log file.
For recurring meetings do not output meetingstatus on exceptions in AS 2.5
==========================================================================
For recurring meetings do not output meetingstatus on exceptions in AS 2.5. Support for
meetingstatus on exceptions was only added in AS 12.1
Updated Out-Of-Office to handle different External messages
===========================================================
Updated Out-Of-Office to handle different External messages. The zimbra settings do not
match up to the typical settings available on most phones so the nearest approximation is
used in both downloading the current settings to the phone and in updating zimbra.
Handle both Autodiscover and Browser Tests when setting the _ua string
======================================================================
Handle both Autodiscover and Browser Tests when setting the _ua string. These are special
cases where there is no DeviceID available. In the case of Autodiscover the Request class
does not exist either.
Allow a default user XML file to be used for all users
======================================================
Allow a default user XML file to be used for all users. This is a feature request from
SourceForge. A new optional config setting
define('ZIMBRA_USER_XML_DEFAULT','default.xml');
has been added to allow for specification of the default file. This will be used for any
user who does not have a specific file named for their account. A user specific file will
be used instead of, and not in addition to, the default one.
Add support for category changing to messages and to _cachedMessageLists
========================================================================
Add support for category changing to messages and to _cachedMessageLists. A recent bug fix
in z-push has allowed for support of categories on messages (mostly used from Outlook)
This change adds a crc32 hash of the Categories (Tags on zimbra) to the cached message lists
modified flag so that changes in Tags/Categories are recognized and outputed to the client.
Incoming Categories/Tags are also now handled for emails. The change affcts the 'mod' flag
in both StatMessage and GetMessageLists. In addition a new attribute CacheSupports has been
added to the cache that will identify the supported flags/categories and will cause a flush
of the CachedMessageLists if the currently supported attributes does not match the cache.
Renamed constructor function from BackendZimbra to __construct
==============================================================
Renamed constructor function from BackendZimbra to __construct as old way has been
deprecated in PHP 7.0
Removed //IGNORE//TRANSLIT from $params array in SendMail
=========================================================
Removed //IGNORE//TRANSLIT from $params array in SendMail as it is generating a warning
when used with the version 2.2.7 or z-push and later
Added new config setting 'ZIMBRA_URL_ALLOW_REDIRECT' for OPEN LDAP auth
=======================================================================
Added new config setting 'ZIMBRA_URL_ALLOW_REDIRECT' for OPEN LDAP auth. It seems that
configuring a domain to have an external OPEN LDAP authorization source results in a reditect
HTML 301 or 302 error getting returned by CURL. Adding the options
curl_setopt($this->_curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($this->_curl, CURLOPT_POSTREDIR, 3);
allows CURL to follow the redirection. The code has been updated so that if the new config
setting is defined, and set to true, the two additional CURL options will be enabled.
Hopefully this will work in all situations. This issue was raised and debugged on the
support forum. https://sourceforge.net/p/zimbrabackend/support-requests/129/
Note, if you want to only cater for HTML error 301 - you can change the code to set the value
for CURLOPT_POSTREDIR to 1, or if you only want to cater for HTML error 302 you can change it
to 2. The chosen value 3 caters for both.
Added SmartFolders flag to cache to detect changes and invalidate cache
=======================================================================
Added SmartFolders flag to cache to detect changes and invalidate cache. If the admin turns
on/off SmartFolders it can cause the folder list for any/every client to become invalid. So
long as the cached folder list is used the device will not realize that the folder list has
changed. Adding the flag to the cache to identify if SmartFolders was on/off at the time the
folder list was cached will allow for detection of cahnges in that setting and as a result
it will allow for faster detection of potential folder list changes.
Added stats field to _folders for use by new FolderStats functionality
======================================================================
Added stats field to _folders for use by new FolderStats functionality in z-push 2.3 and
later. This functionality eliminates the need to setup an exporter to compare the list of
items in the z-push cache against the list of items returned from a query of the folder.
If the stats of the folder have not changed then the exported setup will be skipped.
Added extra GetFolder calls to get data on Shared Folders for FolderStats
=========================================================================
Added extra GetFolder calls to get data on Shared Folders for FolderStats. In the case of
shared folders the fields needed to determine if anything has changed are not all returned
in the refresh block at the beginning of a session. It is necessary to make a GetFolder
request against the sharer's folder tree to return the data. The updated data and stats are
saved to the _folders array and saved to the cache.
Changes Made To Revision 63: z-push-2 version ONLY
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Ensure meetingstatus is output correctly for main and exceptions
- Only output attendeestatus to the meeting organizer
- Change 'Android/5.0' to 'Android/5' as a multi-folder capable _ua string
in order to match Android/5.1 also
- Since z-push 2.2.2 the class StringStreamWrapper has been moved and
pre-included from index.php - Check before including it from old location
- Use new exception constant SyncCollections::HIERARCHY_CHANGED
- Trap additional HTML errors in SoapRequest to prevent removing content
- Add checking for HTML errors in functions Logon and ChangesSink
- Fix ChangeFolder function so create/rename/delete/move all work
- Changed SendMail filter to keep the original body in more cases
- Add neg="1" flag to incoming appointment alarms
- Check for Request class in Logon function before setting client variables
- Removed reference to Request class from function SoapRequest
- Reworked Multi-Folder support adding a new configuration file
- Check for existence of DiffState::RowCmp before calling it
- Use DeviceManager function GetUserAgent if available
Ensure meetingstatus is output correctly for main and exceptions
================================================================
Ensure meetingstatus is output correctly for main and exceptions. The previous
logic was not always outputting it
Only output attendeestatus to the meeting organizer
===================================================
Only output attendeestatus to the meeting organizer. Do not include the attendeestatus
attribute when outputting a received meeting to the device
Change 'Android/5.0' to 'Android/5' as a multi-folder capable _ua string
in order to match Android/5.1 also
========================================================================
Change 'Android/5.0' to 'Android/5' as a multi-folder capable _ua string
in order to match Android/5.1 also. This will hopefully also continue to work with 5.1.x
and 5.2, 5.x if such versions are released in future
Since z-push 2.2.2 the class StringStreamWrapper has been moved and
pre-included from index.php - Check before including it from old location
=========================================================================
Since z-push 2.2.2 the class StringStreamWrapper has been moved and
pre-included from index.php - Check before including it from old location
The new location is in the lib/utils folder whereas it previously resided
in the include folder. The code should now work for either case
Use new exception constant SyncCollections::HIERARCHY_CHANGED
=============================================================
Use new exception constant SyncCollections::HIERARCHY_CHANGED which was added by z-push
in response to a ticket raised against their LoopDetection. A series of folder name
changes in quick succession would trigger a full resync of the client. Using this new
exception status does not log an error and thus does not trip the LoopDetection code
Trap additional HTML errors in SoapRequest to prevent removing content
======================================================================
Trap additional HTML errors in SoapRequest to prevent removing content, This is an
issue related to the use of the zimbra nginx proxy. When the system is partially up
(starting/stopping) nginx responds with an HTML error page instead of the expected
SOAP response from the backend. Code has been added to trap these error pages. This
should prevent the backend from reporting to z-push that only one folder (the FakeOutbox)
remains for the device, ad doing so was causing the device to wipe all data
Add checking for HTML errors in functions Logon and ChangesSink
===============================================================
Add checking for HTML errors in functions Logon and ChangesSink. This uses the newly
added HTML error trapping to slow down the retry login of the device by sleeping for a
minute before retrying or returning control to the z-push ChangesSink engine. This is
the same behaviour as is already being used to address CURL errors when zimbra is
completely down
Fix ChangeFolder function so create/rename/delete/move all work
===============================================================
Fix ChangeFolder function so create/rename/delete/move all work. Rename always worked
but folder create has been broken for some time - on Outlook 2013 this would manifest
itself in a second copy of the folder getting created with a (1) extension on the name.
Move and Delete were not previously implemented. All four functions should now work.
Changed SendMail filter to keep the original body in more cases
===============================================================
Changed SendMail filter to keep the original body in more cases. Now the selection is
not based on identified content body types but rather on whether there is any need for
rebuilding the HTML/Text bodies as is required for smart-forwarding for instance. Now
if there is no specific need to reformat the email, the body will be re-used as-is.
Add neg="1" flag to incoming appointment alarms
===============================================
Add neg="1" flag to incoming appointment alarms as otherwise the zimbra desktop reminder
alert would occur at the specified number of minutes after the meeting instead of before
it. Without this flag, dismissing a reminder on the phone would update the meeting on
zimbra removing the existing neg=true setting, and cause the reminder on recurring
meetings to move to after the meeting had started. Dismissing the reminder on the zimbra
desktop would not alter the meeting in the calendar. Dismissing one of these bad late
reminders on the phone should now restore the flag and correct the future reminders.
Check for Request class in Logon function before setting client variables
=========================================================================
Check for Request class in Logon function before setting client variables so that calls
from Autodiscover do not result in errors
Removed reference to Request class from function SoapRequest
============================================================
Removed reference to Request class from function SoapRequest as it is also called by
Autodiscover. Changed the local variables $deviceIdForMailboxLog added to multiple
functions in version 62.3 to be a global $_deviceIdForMailboxLog, and set it in Logon.
Reworked Multi-Folder support adding a new configuration file
=============================================================
Reworked Multi-Folder support adding a new configuration file to store the list of
supported UserAgent matching strings, and the switches to enable/disable multi-folder
support for each of the non-message folder types (appointment, contact, task, note)
The file named zimbraMultiFolderUAs.php should be placed in the backend/zimbra folder
Lines in the file will have the format
$multiFolderList[] = array( 'ua'=>'Apple', 'message'=>true, 'contact'=>true, 'appointment'=>true, 'task'=>true, 'note'=>true );
where the 'ua' is a partial UserAgent string. It will match to any UserAgent that contains
the text 'Apple' in this example - so it will match with Apple-iPad..., Apple-iPhone...,
etc.
At DEBUG log level, the log file will contain an entry that will show the configuration
that was set for the device including which 'ua' string it matched with - for example
06/08/2015 14:57:56 [ 4514] [DEBUG] [myUser] [appl....] Zimbra->Logon(): Multi-Folder support configured using [Apple] with settings Calendar [SUPPORTED], Contacts [SUPPORTED], Tasks [SUPPORTED], Notes [SUPPORTED]
Each folder type will be identified as either SUPPORTED, NOT SUPPORTED, or GLOBALLY DISABLED (in the case of the feature being disabled in the config.php file)
If no matching 'ua' is found for a device, then the default configuration will be used
turning off all multi-folder capabilities. This will log as follows
06/08/2015 12:21:13 [ 7811] [DEBUG] [myUser] [myDevice] Zimbra->Logon(): Multi-Folder support configured using [DEFAULT] with settings Calendar [NOT SUPPORTED], Contacts [NOT SUPPORTED], Tasks [NOT SUPPORTED], Notes [NOT SUPPORTED]
Check for existence of DiffState::RowCmp before calling it
==========================================================
Check for existence of DiffState::RowCmp before calling it as an upcoming change in z-push
is going to remove that function. For now it is necessary to maintain the check in case
users are running the latest zimbra backend with older z-push versions
Use DeviceManager function GetUserAgent if available
====================================================
Use DeviceManager function GetUserAgent if available to fetch the UserAgent string for a
device. This function will be added in z-push 2.2.4 or 2.3 and should prove more reliable
than using Request GetUserAgent as that function was unable to return UserAgent strings
for Windows 10 Mail and some other devices that only supply their UserAgent in an initial
Settings command when they start to sync first.
Changes Made To Revision 62: z-push-2 version ONLY
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Add 'Android/5.0' as a multi-folder capable _ua string
- Report actual UserAgent, partial DeviceID, and IP Address in headers
- Remove X-Mailer-Connector header from SendMail
- Output meetingstatus on Exceptions too
- Output recurrence type 1 for zimbra "DAI"ly appointments that are weekly
- Added X-Forwarded-For HTTP header to CURL options
- Wrap subject with htmlspecialchars in 3 places it was overlooked
- Add check for class ZPushAutodiscover to initial host version check
- Additional check added to functions GetAttachmentData and
ItemOperationsGetAttachmentData to allow for Sub-Folder of Shared folder
- Disable document access if Class SyncDocumentLibraryDocument does not exist
- Fix in MakeXMLTree for case where tag has no attributes
- Add required use of new curl_file_create function for PHP 5.5 and later
- Output the Organizer Name/Email on Appointments if available
Add 'Android/5.0' as a multi-folder capable _ua string
======================================================
Add 'Android/5.0' as a multi-folder capable _ua string. Android 5.0 is now
capable of synching multiple calendar folders. The string Andrroid/5.0 has been
added to the multi-folder support checks.
Report actual UserAgent, partial DeviceID, and IP Address in headers
====================================================================
Report actual UserAgent, partial DeviceID, and IP Address in headers for zimbra mailbox.log
This will help with tracing connections for troubleshooting purposes as the log can be
filtered by a particular Device ID to isolate connections from that device, and will also
allow for identifying the devices that are connecting and when they are doing so.
NOTE: Only the last 6 characters of the DeviceID will be included.
Remove X-Mailer-Connector header from SendMail
==============================================
Remove X-Mailer-Connector header from SendMail as the UserAgent and partial DeviceID
are included by the server on the X-Mailer header (at least from zimbra 8.5) The X-Mailer
header in the sent email now looks like the following
X-Mailer: Zimbra 8.6.0_GA_1153 (SAMSUNG-GT-I9505/101.40402(...3470de) devip=192.168.220.252 ZPZB/61.2)
showing the server version, client UserAgent, last 6 characters of the device ID, the IP
address of the sending device, and finally the Z-Push Zimbra Backend (ZPZB) version in use
Output meetingstatus on Exceptions too
======================================
Output meetingstatus on Exceptions too. Up to now this has only been included on the
parent meeting/appointment.
Output recurrence type 1 for zimbra "DAI"ly appointments that are weekly
========================================================================
Output recurrence type 1 for zimbra "DAI"ly appointments that are weekly. Zimbra uses a
strange set of recurrence patterns. On updating meetings that zimbra defines as DAILY
that occur multiple times per week, and repeat weekly, devices have been seen to drop
parts of the recurrence pattern. This results in zimbra dropping all exceptions to the
original meeting/appointment. Outputting the type as WEEKLY (1) instead allows
modifications to work correctly.
Added X-Forwarded-For HTTP header to CURL options
=================================================
Added X-Forwarded-For HTTP header to CURL options. Based initially on a feature request
https://sourceforge.net/p/zimbrabackend/feature-requests/11/
but modified so as to make it show the DeviceID regardless of whether there is an Nginx
proxy in place for zimbra or not.
Wrap subject with htmlspecialchars in 3 places it was overlooked
================================================================
Wrap subject with htmlspecialchars in 3 places it was overlooked. There were a number of
instances where the Subject of a meeting/appointment was not wrapped in htmlspecialchars
encoding. One of these lead to the inability to cancel an appointment with an ampersand
character in the name.
Add check for class ZPushAutodiscover to initial host version check
===================================================================
Add check for class ZPushAutodiscover to initial host version check as otherwise the code
will fail to load for AutoDiscover, and will report that it does not work with z-push
version prior to 2.0
Additional check added to functions GetAttachmentData and
ItemOperationsGetAttachmentData to allow for Sub-Folder of Shared folder
========================================================================
Additional check added to functions GetAttachmentData and ItemOperationsGetAttachmentData
to allow for Sub-Folder of Shared folder. This issue was brought to light in a ticket
http://sourceforge.net/p/zimbrabackend/bugs/51/ and resulted in a failure to download
attachments from sub-folders of shared folders. Previous testing had only ever tested top
level shared folders.
Disable document access if Class SyncDocumentLibraryDocument does not exist
===========================================================================
Disable document access if Class SyncDocumentLibraryDocument does not exist. This will
prevent warnings about unsupported folder types. It should also allow the code to work as
is once the core z-push class files are added.
Fix in MakeXMLTree for case where tag has no attributes
=======================================================
Fix in MakeXMLTree for case where tag has no attributes. This addresses a warning that
only occurs when SmartFolders are NOT in use.
Add required use of new curl_file_create function for PHP 5.5 and later
=======================================================================
Add required use of new curl_file_create function for PHP 5.5 and later. This issue was
highlighted in ticket https://sourceforge.net/p/zimbrabackend/support-requests/105/ and
resulted in SendMail failing due to the new version of PHP disallowing the @ operator
on the filename.
Output the Organizer Name/Email on Appointments if available
============================================================
Output the Organizer Name/Email on Appointments if available. Currently when accepting an
invitation by email on iOS sevices the organizer is displayed as it is available from the
email. But once the appointment is accepted, the organizer does not save with it, and it
is no longer displayed on the iOS device. Please note that it is still not possible to
send an updated status response for a meeting from the iOS devices. They do not return a
MeetingResponse command to the server. They attempt to upload a modified appointment to
the server and send an email directly to the organizer. For some reason zimbra does not
update the meeting based on the email it receives.
Changes Made To Revision 61: z-push-2 version ONLY
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Remove forcing CURLOPT_SSLVERSION to 3 (to avoid SSLv3 POODLE issue)
- Fix SendMailSenderFix email address for condition where no from header
and full email address used as username
- Throw exception on SOAP FAULT - service.AUTH_EXPIRED to force re-auth
- Fix initialization of _userFolderTypeActive based on GetInfoResponse
- Add ZIMBRA_DISABLE_DOCUMENTS setting to Config File notes below
- Added function GetUserDetails needed for AutoDiscover feature
- Fix check for zimbraPrefFromDisplay and zimbraPrefFromAddress
- Use configured zimbraPrefFromAddress as sender email address if different
from account name
- Fix clear SendAsNameOverride, SendAsEmailOverride, ServerInviteReply from
cache
Remove forcing CURLOPT_SSLVERSION to 3 (to avoid SSLv3 POODLE issue)
====================================================================
Remove CURLOPT_SSLVERSION to 3 - This setting was added in Release 59 following feedback
from a user who had issues with CURL in the absense of this setting. However with the
recent discovery of the POODLE SSLv3 vulnerability it is no longer desirable to force the
use of SSLv3. - as this is preventing negotiation of TLS. Please see forum post
https://community.zimbra.com/collaboration/f/1906/t/1135430 for more information.
Fix SendMailSenderFix email address for condition where no from header and full email
address used as username
=====================================================================================
Fix SendMailSenderFix email address for condition where no from header and full email
address used as username. In this case, the code was incorrectly setting the "User Name"
as the email address.
Throw exception on SOAP FAULT - service.AUTH_EXPIRED to force re-auth
=====================================================================
Throw exception on SOAP FAULT - service.AUTH_EXPIRED to force re-auth. If the zimbra
session expires during a sync period, the ChangesSink loop will continually return the
Auth Expired fault until the ping session expires. This will cause the loop to miss any
new emails/events it should otherwise notify to the device, and will flood the log with
errors. Throwing an exception forces z-push to drop the session and return an error to
the device requesting it to try the command again. The new connection will issue a new
AuthRequest and get a new session.
Fix initialization of _userFolderTypeActive based on GetInfoResponse
====================================================================
Fix initialization of _userFolderTypeActive based on GetInfoResponse using the correct
default for every different folder type. Previously all were using the default for the
'message' folder type - which is always set to true.
Add ZIMBRA_DISABLE_DOCUMENTS setting to Config File notes below
===============================================================
Add ZIMBRA_DISABLE_DOCUMENTS setting to Config File notes below. This setting disables
the handling of Briefcase folders. This is needed if the z-push enhancements to handle
Document Libraries are not installed. Those files have been contributed to the z-push
contrib-zp2 branch but have not been evaluated for merging with the main branch as yet.
Added function GetUserDetails needed for AutoDiscover feature
=============================================================
Added function GetUserDetails needed for AutoDiscover feature. This functionality still
needs testing.
Fix check for zimbraPrefFromDisplay and zimbraPrefFromAddress
=============================================================
Fix check for zimbraPrefFromDisplay and zimbraPrefFromAddress in function GetUserInfo()
Use configured zimbraPrefFromAddress as sender email address if different from account name
===========================================================================================
Use configured zimbraPrefFromAddress as sender email address if different from account name
The account name is the default email address for the user. If they have selected a
configured alias as their prefeerred from address in the Web Client, then use this address
from the zimbra backend too.
Fix clear SendAsNameOverride, SendAsEmailOverride, ServerInviteReply from cache
===============================================================================
Fix clear SendAsNameOverride, SendAsEmailOverride, ServerInviteReply from cache when Folder
cache is re-built and these settings are no longer present as directives
Changes Made To Revision 60: z-push-2 version ONLY
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Save folder owner at Setup for shared folders, and use folder owner as
organizer when creating appointments in a shared calendar folder
- Add new function isZimbraObjectInSyncInterval() to check if an item being
changed/deleted by a device is within the devices sync window.
- Assign default of 'Unavailable' to _changeToken if it is not already set
- Initialise _cacheChangeToken to 'NotSet' if cache is not used
- Do not use absFolderPath for shared folder as it is path in other mailbox
- Rename _deviceAgent to _ua
- Add 'Android/4.4' as a multi-folder capable _ua string
- Removed uninitialized variable _num_soap_calls from function SoapRequest()
- Add authToken to header for NoOpRequest() calls for Nginx traversal
- Don't call isZimbraObjectInSyncInterval() from ChangeMessage if id is blank
- Add check for parentid '0' in GetFolder() before identifying it as a
special folder - Inbox, Sent, Trash, Drafts, Outbox
- Change the eol in SendMail from "\n" to "\r\n" as Outlook 2013 was unable
to display Sent Items correctly with "\n"
Save folder owner at Setup for shared folders, and use folder owner as organizer
================================================================================
Save folder owner at Setup for shared folders, and use folder owner as organizer
when creating appointments in a shared calendar folder - and also when moving
appointments (not meetings) between shared and personal folders and vice-versa
Add new function isZimbraObjectInSyncInterval() to check if an item being
changed/deleted by a device is within the devices sync window.
=========================================================================
Add new function isZimbraObjectInSyncInterval() to check if an item being
changed/deleted by a device is within the devices sync window. HTC devices
have been seen to send Delete commands for items that have expired from
their sync window instead of ignoring them and letting the server tell it
to remove them
Assign default of 'Unavailable' to _changeToken if it is not already set
========================================================================
Assign default of 'Unavailable' to _changeToken if it is not already set. This
is to avoid errors when checking to see if value has changed
Initialise _cacheChangeToken to 'NotSet' if cache is not used
=============================================================
Initialise _cacheChangeToken to 'NotSet' if cache is not used. This is to avoid
errors when checking to see if value has changed
Do not use absFolderPath for shared folder as it is path in other mailbox
=========================================================================
Do not use absFolderPath for shared folder as it is path in other mailbox. The
absFolderPath is not returned by zimbra for shared folders. It is necessary to
build up the path step by step from the root folder.
Rename _deviceAgent to _ua
==========================
Rename _deviceAgent to _ua. Just to shorten name and make it consistent with
reporting in the log.
Add 'Android/4.4' as a multi-folder capable _ua string
======================================================
Add 'Android/4.4' as a multi-folder capable _ua string. Android 4.4 is now
capable of synching multiple calendar folders. The string Andrroid/4.4 has been
added to the multi-folder support checks.
Removed uninitialized variable _num_soap_calls from function SoapRequest()
==========================================================================
Removed uninitialized variable _num_soap_calls from function SoapRequest(). This
variable is not needed
Add authToken to header for NoOpRequest() calls for Nginx traversal
===================================================================
Add authToken to header for NoOpRequest() calls for Nginx traversal. This tag
was omitted when copying the test code change into the main branch.
Don't call isZimbraObjectInSyncInterval() from ChangeMessage if id is blank
===========================================================================
Don't call isZimbraObjectInSyncInterval() from ChangeMessage if id is blank. This
will happen when a new object is created on the device and sent to the server for
the first time. It will not already exist in the sync window by definition and
looking for it will generate an invalid SOAP request which will fail.
Add check for parentid '0' in GetFolder() before identifying it as a special folder
===================================================================================
Add check for parentid '0' in GetFolder() before identifying it as a special folder
- Inbox, Sent, Trash, Drafts, Outbox. This is to avoid having linked or backup
copies of Inbox for example getting mistaken for the real one.
Change the eol in SendMail from "\n" to "\r\n" as Outlook 2013 was unable to display
Sent Items correctly with "\n"
====================================================================================
Change the eol in SendMail from "\n" to "\r\n" as Outlook 2013 was unable to display
Sent Items correctly with "\n"
Changes Made To Revision 59:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Add folders to local cache to reduce server load on Logon, especially on
initial sync where the device will repeatedly connect to request more data
while at the same time few if any changes will be made to the server content
- Added setting CURLOPT_SSLVERSION, 3 to avoid errors on some platforms
- Improved publicURL matching and reduced associated logging
- Changed RowCmp() fix in diffstate.php to allow testing for presence of fix
- Added test for RowCmp() fix to Logon() function. If the fix is not detected
an error is logged to z-push-error.log
- Fix adding appointment to shared calendar
- Fix changes to shared calendars notified on linkid instead of zimbrafolderid
Note zimbra considers these non-critical so does not notify immediately. The
notification will be returned when the current ChangesSink expires
- Remove restriction on moving items between personal/shared folders as
fix for notifications in this release solves the issue
- Make sure dayofweek/dayofmonth values are set before outputting appointments
- Make sure exceptions arrays exist before foreach in ChangeMessage (APPT)
- Added loop to check for changes to both folders and links in ChangesSink
- Added User Agent string to soap context headers
- Removed .z-push-2 extension from revision as other versions are deprecated
- Added extra checks for notify block in ChangesSink to catch some previously
missed notifications, and to identify item Moves
- Set default timeout to 60 seconds on ChangesSink
- Various code clean-ups
Added to repackaged zimbra59.tgz on 19-Mar-2014
- Remove the & references from calls to ChangesSinkNotify()
Add folders to local cache to reduce server load on Logon
=========================================================
Add folders to local cache to reduce server load on Logon, especially on initial sync
where the device will repeatedly connect to request more data while at the same time few
if any changes will be made to the server content
Added setting CURLOPT_SSLVERSION, 3 to avoid errors on some platforms
=====================================================================
Added setting CURLOPT_SSLVERSION, 3 to avoid errors on some platforms. User of Ubuntu
reported getting curl exec error 35 (an SSL protocol error) in the z-push-error.log file
and suggested this fix. A quick google search confirmed that this error is quite common
on Ubuntu in particular.
As SSLv3 was released in 1999 it is not likely there is any installation out there that
doesn't have it installed as standard, so it makes sense to add this setting.
Improved publicURL matching and reduced associated logging
==========================================================
Improved publicURL matching and reduced associated logging. Comparisons will now ignore
:443 on https host URLs and :80 on http host URLs. The number of log statements has been
reduced and most have been raised to DEBUG level leaving just the override notification
lines at WARN level. This should reduce the noise in production logs.
Changed RowCmp() fix in diffstate.php to allow testing for presence of fix
==========================================================================
Changed RowCmp() fix in diffstate.php to allow testing for presence of fix so that it is
obvious to admins that the change is needed. The change change required in
z-push/lib/default/diffbackend/diffstate.php in the function RowCmp() is as follows
static public function RowCmp($a, $b) {
// TODO implement different comparing functions
// return $a["id"] < $b["id"] ? 1 : -1;
// VJS - For ZIMBRA backend - need to use STRING comparisons - and allow strcmp to return 0 for equal
// Zimbra->Logon() function will test compare two equal strings to verify presence of this fix.
return strcmp( $b["id"], $a["id"] ) ;
}
NOTE: the order of $b and $a has been switched in addition to removing the conditional logic
If the call to RowCmp() from the Zimbra->Logon function does not return 0 the fix has not been
applied as the baseline z-push fix will always return either -1 or 1
Added test for RowCmp() fix to Logon() function
===============================================
Added test for RowCmp() fix to Logon() function. This has been added as a result of the number of
implementations that are put into production without the fix only for clients to experience
looping problems and in some cases duplication of contacts/tasks/appointments.
If the fix is not detected an error is logged to z-push-error.log
Fix adding appointment to shared calendar
=========================================
Fix adding appointment to shared calendar. This corrects a problem where a user selects a shared
calendar while creating an appointment on any device that supports multiple calendars. On saving
the calendar event it would get wrongly saved to the user's default calendar folder on zimbra
instead of to the shared calendar. The incorrect event would then get synched back to the device
resulting in duplicate events.
Fix changes to shared calendars notified on linkid instead of zimbrafolderid
============================================================================
Fix changes to shared calendars notified on linkid instead of zimbrafolderid. This was causing
the ChangesSink to ignore them.
Note zimbra considers notifications on other people's folders to be non-critical so does not
notify the user about them immediately. The notification will be returned when the current
ChangesSink expires. The duration of the ChangesSink is dictated by the z-push config setting
define('PING_INTERVAL', 60);
This should not be changed below the default of 30 seconds. I recommend it is set to 60 seconds.
Remove restriction on moving items between personal/shared folders
==================================================================
Remove restriction on moving items between personal/shared folders as fix for notifications in
this release solves the issue that was requiring that the restriction be put in place
Make sure dayofweek/dayofmonth values are set before outputting appointments
============================================================================
Make sure dayofweek/dayofmonth values are set before outputting appointments. For appointments
created on Web Client as Weekly/Monthly where the recurrence pattern is not clicked to customise
Zimbra would omit saving the dayofweek/dayofmonth. But modifications to these appointments would
have the fields set so the recurrence pattern would never match which in turn would cause
exceptions to get stripped from the appointment post-modification. May also happen with
appointments created using CalDAV in some circumstances.
Make sure exceptions arrays exist before foreach in ChangeMessage (APPT)
==========================================================================
Make sure exceptions arrays exist before foreach in ChangeMessage (APPT) - if either the input
appointment or the pre-modified appointment had no existing exceptions then the foreach loops
would generate a WARNing in the log
Added loop to check for changes to both folders and links in ChangesSink
==================================================================
Added loop to check for changes to both folders and links in ChangesSink. The code was checking
if any of them had been updated and then processing updates for folders if any were reported,
otherwise, it was processing links. If both were reported, links were not getting processed.
Added User Agent string to soap context headers
===============================================
Added User Agent string to soap context headers - this will log to the zimbra log file
(/opt/zimbra/log/mailbox.log) and make it obvious where the soap calls came from. For example :-
2014-03-14 00:00:00,655 INFO [btpool0-1789://myServer/service/soap/]
[name=myUser;mid=10;ip=192.168.123.123;ua=zimbrabackend/58.6;] soap - GetInfoRequest elapsed=1
Removed .z-push-2 extension from revision as other versions are deprecated
==========================================================================
Removed .z-push-2 extension from revision as other versions are deprecated. This just shortens
the reported version in the logs
Added extra checks for notify block in ChangesSink to catch some previously missed notifications
================================================================================================
Added extra checks for notify block in ChangesSink to catch some previously missed notifications,
and to identify item Moves where the notification from the receiving folder is not always caught
before the notification block is returned to the ChangesSink process.
Set default timeout to 60 seconds on ChangesSink
================================================
Set default timeout to 60 seconds on ChangesSink. z-push original was 30 seconds. This value is
normally overridden by the config.php setting
define('PING_INTERVAL', 60);
Various code clean-ups
======================
Various code clean-ups. Tidying up variable names/logging/etc.
Added to repackaged Release59 on 18-Mar-2014
Remove the & references from calls to ChangesSinkNotify()
=========================================================
Remove the & references from calls to ChangesSinkNotify(). These were left in place accidentally
when the new function definition was copied/pasted to create the calls to the function.
Changes Made To Revision 58:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Fix Logoff to only call EndSessionRequest if session was _connected and
also add try/catch to ensure no exceptions are throw back to index.php
- Change flagged memory amount to 10M in ReportMemoryUsage
- Filter out Content-Transfer-Encoding header in SendMail
- Fix content-transfer-encoding substring length (change from 13 to 26)
- Add authToken back into SoapRequest header as it seems that passing it as
a cookie does not suffice in all scenarios
- Implement Flags support to sync set/unset flags both ways. Also sends
updated LastVerbExecuted
- Removed default value "=false" from $contentParameters parameter on
functions MoveMessage, ChangeMessage, DeleteMessage and SetReadFlag
- Change GetDocumentLibrary to use exact filename given (remove str_replace)
Fix Logoff to only call EndSessionRequest if session was _connected and also add try/catch
==========================================================================================
Fix Logoff to only call EndSessionRequest if session was _connected. This is to avoid calling
EndSessionRequest when an incorrect userid/password combination is entered. This will avoid
generating errors. Also add try/catch to ensure no exceptions are throw back to index.php as
these might break the returns from z-push to the device and interrupt provisioning for example
Change flagged memory amount to 10M in ReportMemoryUsage
========================================================
Change flagged memory amount to 10M in ReportMemoryUsage as the normal memory usage is now
just above 9M there is no point in flagging the majority of all calls.
Filter out Content-Transfer-Encoding header in SendMail
=======================================================
Filter out Content-Transfer-Encoding header in SendMail to avoid having multiple conflicting
headers in the sent email. This issue cropped up for android 4.4 clients that started sending
emails base64 encoded. The reworked SendMail in Release57 was not filtering out this header
so when the mail_mime tools rebuilt the MIME body and added the default quoted-printable
header and content, the receiving email system was getting confused. It had the first
Content-Transfer-Encoding header telling it the body was base64 encoded - but the actual body
was encoded as quoted-printable. When the receiving email client decoded it as base64 it
produced garbage.
Fix content-transfer-encoding substring length (change from 13 to 26)
=====================================================================
Fix content-transfer-encoding substring length (change from 13 to 26). The initial check-in of
Release 57.1 had an incorrect substring length parameter which resulted in the test always
failing
Add authToken back into SoapRequest header
==========================================
Add authToken back into SoapRequest header as it seems that passing it as a cookie does not
suffice in all scenarios. Release 57 had switched the auth token to a COOKIE based on feedback
from zimbra on the forums. It turns out that for some zimbra releases/configurations this method
does not work. It is still reuired to pass the new COOKIE in order to traverse an nginx proxy
but it is also necessary to pass the authToken in the old way - the SoapRequest header - in
addition to passing the COOKIE
Implement Flags support to sync set/unset flags both ways. Also sends updated LastVerbExecuted
==============================================================================================
Implement Flags support to sync set/unset flags both ways. Also sends updated LastVerbExecuted
This feature was introduced in the as12.1 branch but could not be migrated as-was over to z-push-2
As a result it has been largely non-functional in z-push-2. This release implements the feature in
a new way that will allow it to work. The last verb executed will also be sent to the device if it
changes subject to the zimbra web client rules. Replied takes precedence over Forwarded. So, if
you forwarded a message that forwarded verb would get sent to the device. If you subsequently
replied to the same message - the replied verb would get sent to the device. If you performed the
steps in the opposite order the replied verb will not be overridden as it takes precedence.
Removed default value "=false" from $contentParameters parameter on functions MoveMessage,
ChangeMessage, DeleteMessage and SetReadFlag
==========================================================================================
Removed default value "=false" from $contentParameters parameter on functions MoveMessage,
ChangeMessage, DeleteMessage and SetReadFlag as they are throwing PHP Fatal errors on some
platforms. These defaults were needed to provide compatibility with z-push releases prior to
2.0.6RC. The removal of them means that this backend file is no longer compatible with those
z-push releases prior to 2.0.6RC
Change GetDocumentLibrary to use exact filename given (remove str_replace)
==========================================================================
Change GetDocumentLibrary to use exact filename given (remove str_replace). This prevents
selected file from returning a not-found error.
Changes Made To Revision 57:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- SmartFolders are now enabled by default - disable them to use XML files
- Changed to pass session ID as CURLOPT_COOKIE for nginx traversal
- Add NoOpRequest to return session ID/session refresh through nginx
- Added support for Syncing StickyNotes from the device to specially named
Tasks folders "Notes" (primary) and "Notes?????" for additional folders
- Streamlined SendMail function to better handle uneditable emails and pass
through all mime headers received from the client
- Added pass-through in SendMail for S/Mime signed and encrypted emails
- Moved From header handling from SendMail to new SendMailSenderFix function
- Added idents to GetInfoRequest, and added lookup of preferred FromName
against the different identities
- Added function GetDocumentLibrarySearchResults to search Briefcase folders
- Added GetLibraryDocument to return contents of Briefcase documents
- Added MaxRange check in each ...SearchResults function
- Check for existence of class SyncResolveRecipientResponse and return a
protocol error response if it is not found. This will prevent errors in the
short term until the Z-Push core code has the appropriate support
- Check publicURL and soapURL returned for user - log warning on differences
- Check for existence of messages in search response before counting them
- Check and remove "<" and ">, " from $searchquery in GetGALSearchResults
- Added new configuration option ZIMBRA_SERVER_INVITE_REPLY (assumed false)
to control the sending of Calendar Invites/Replies from the zimbra server
as most devices now send these directly. This will avoid duplicate invites/
replies getting sent to invitees/attendees/organizers
- Added SmartFolders directive serverinvitereply to allow overriding the
default ZIMBRA_SERVER_INVITE_REPLY setting on a per device basis
- Removed all references to $folderTypeInactive to simplify code. Now set
folder include parameter to false for any folder whose active setting is
false.
- Check existence of Message Fragment ['fr'] before setting preview
- Added check for Jetty error 'Error 503 Service Unavailable' to SoapRequest
- Fixed MeetingRequest organizer so email forwards/replies do not change it
- Added throw FolderSync Exception if folder not found in index
- Added error handling to file upload in SendMail
- Fixed ChangesSink to detect folder changes in monitored linked folders
- Added set $_userFolderTypeActive based on zimbra prefs in GetInfoRequest
- Added Cleanup expired cachedMessageLists on Logoff.
- Fix QuickAdd yearly appointment - Make type 5 and extract month/dayofmonth
- Fixed DeleteMessage(appt) to call CancelAppointmentRequest if organizer
- Fixed MoveMessage(appt) to call CancelAppointmentRequest if organizer
- Added dynamic rate limiter _soapDelayMicroSeconds to SoapRequest
- Change to not output the Briefcase folders in the Folder List
- Fixed to output recurring appointment exceptions(deletions) from CalDAV
- Reorganized GetMessage (appointment) to minimise data send to device
- Added settings for _deviceMultiFolderSupport for Outlook andWindowsMail
- Fixed interaction between _deviceMultiFolderSupport and Virtual folders
- Tidied up various log messages
Added to repackaged Release 57 of 19-Nov-2013
- Add check in SendMailSenderFix for full address as DisplayName
- Fix ZIMBRA_DISABLE_URL_OVERRIDE logic. If "true" do not override
SmartFolders are now enabled by default - disable them to use XML files
=======================================================================
SmartFolders are now enabled by default - disable them to use XML files.
Since there are more and more people using the backend now for the first time
and most if not all functionality of the XML files can be accomplished using
the SmartFolders capabilities, it has been decided that SmartFolders will now
be enabled by default - unless specifically disabled in teh config.php file.
This is a change from the previous behaviour where SmartFolders were disabled
by default. Only installations that do not have a ZIMBRA_SMART_FOLDERS
directive in their config.php file will be affected.
Changed to pass session ID as CURLOPT_COOKIE for nginx traversal
================================================================
Changed to pass session ID as CURLOPT_COOKIE for nginx traversal. Following
extensive troubleshooting and questions back and forth with a Zimbra employee
on the Developer forum it finally came to light that the Auth Token MUST be
passed as a cookie to CURL in order for the nginx proxy to pass the request
back to the correct mail store for the user account. Passing the token in the
SOAP header does not steer the request to the correct mail store. This fix
stops the folder list/emails from appearing/disappearing/re-appearing/etc.
Add NoOpRequest to return session ID/session refresh through nginx
==================================================================
Add NoOpRequest to return session ID/session refresh through nginx. In
addition to requiring the passing of the Auth Token as a cookie, it also came
to light that a session context will only be created for the user IF the
nginx round-robin happened to send the AuthRequest to the mail store that
holds the user's account. If the resuest was passed to any other mail store,
it will return an Auth Token but no session context. In this case, another
request needs to be sent back to the server (passing the AuthToken as a cookie
to steer it to the correct mail store) and this request will be able to create
a session context. Logic has been added to send a NoOpRequest back to the
server in the event that a session context is not established by the initial
AuthRequest
Added support for Syncing StickyNotes from the device to specially named
Tasks folders "Notes" (primary) and "Notes?????" for additional folders
========================================================================
Added support for Syncing StickyNotes from the device to specially named
Tasks folders "Notes" (primary) and "Notes?????" for additional folders.
iOS7 has introduced Notes sync to the mail client and, as Tasks are the only
objects that can be created/deleted with ease on zimbra in any folder without
requiring to/from email address, this is the most logical place to store the
synched Notes.
The user must create the tasks folders on the server for them to show up on
the iOS7 device. All "Notes" folders must be named starting with the word
"Notes". A folder called "Notes" with no further characters after it in the
name will be treated as the primary Notes Folder.
Streamlined SendMail function to better handle uneditable emails
================================================================
Streamlined SendMail function to better handle uneditable emails and pass through
all mime headers received from the client. There are many emails from the client
that ought not be broken apart and re-packaged such as calendar invites/replies,
S/MIME messages/ms-tnef encoded messages. The code has been reorganized to much
better handle all of these message types, while also passing through all message
headers received from the client. In these cases just the From: header is
potentially changed according to the Display Name matching rules, and the
enforcement of valid sender email addresses.
Added pass-through in SendMail for S/Mime signed and encrypted emails
=====================================================================
A user reported an issue with the backend not passing through S/MIME signed and/or
encrypted messages correctly. With the help of some debug logs from the user I was
able to identify the format of the headers of the affected messages, and make
changes to allow them to pass through correctly. This effort involved a major
re-write of the SendMail function but it is now much more logically structured
and easier to follow
Moved From header handling from SendMail to new SendMailSenderFix function
==========================================================================
As part of the SendMail rewrite, I moved the code to fix the Sender Name out of
the main function into a function of it's own. Now all decisions on what name
to use are in a single easy to follow function.
Added idents to GetInfoRequest
==============================
Added idents to GetInfoRequest, and added lookup of preferred FromName against the
different identities returned for the user. This means if the user has set up
multiple personas, each with a different email address and Display Name, the code
will use the email address/login id from the device to pick the most appropriate
Sender Display Name to match it.
Added function GetDocumentLibrarySearchResults to search Briefcase folders
==========================================================================
Added function GetDocumentLibrarySearchResults to search Briefcase folders, and
sent supporting z-push code to z-push team for inclusion in the product. This
functionality requires z-push to have the submitted code/or a similar code block
in order to expose the functionality. Still awaiting feedback on potential merging
of this function
If made available, it will allow a user of a client with Sharepoint document support
to request documents from zimbra Briefcase folders by searching from the root folder
"zimbra". This search will return the list of the user's Briefcase folders, and any
non-"zimbra document" items in the folder.
Zimbra Documents are not supported at this time, as there is no soap API available
to download them. There are several zimbra bugzilla bugs open on this topic.
Added GetLibraryDocument to return contents of Briefcase documents
==================================================================
Added GetLibraryDocument to return contents of Briefcase documents requested
following a successful DocumentLibrarySearch.
Added MaxRange check in each ...SearchResults function
======================================================
Added MaxRange check in each ...SearchResults function according to the latest
ActiveSync specifications. If the client requests too many search results, the
server will return a protocol error.
Check for existence of class SyncResolveRecipientResponse
=========================================================
Check for existence of class SyncResolveRecipientResponse and return a protocol
error response if it is not found. This will prevent errors in the short term until
the Z-Push core code has the appropriate support. This class is part of code
submitted to the Z-Push team by me to properly implement the ResolveRecipients
function. Until it is put in place, the ResolveRecipients function will always fail
so it is best to fail nicely.
Check publicURL and soapURL returned for user - log warning on differences
==========================================================================
As part of troubleshooting an issue with a multi-server user, code was added to
check the publicURL and soapURL returned for user, and to log a warning on any
time the soapURL does not begin with the publicURL.
Check for existence of messages in search response before counting them
=======================================================================
Check for existence of messages in search response before counting them. This is a
cosmetic fix to prevent a warning in the event that no results were returned from
a search.
Check and remove "<" and ">, " from $searchquery in GetGALSearchResults
=======================================================================
If the user presses the RETURN key on an device while entering an invitee email
address, the device adds a "<" before the text they have entered, and ">, " after it.
This breaks the parsing in the search backend. A change has been added to check for
and remove those characters if necessary.
Added new configuration option ZIMBRA_SERVER_INVITE_REPLY (assumed false)
=========================================================================
Added new configuration option ZIMBRA_SERVER_INVITE_REPLY (assumed false)
to control the sending of Calendar Invites/Replies from the zimbra server
as most devices now send these directly. This will avoid duplicate invites/
replies getting sent to invitees/attendees/organizers
As most modern devices now send the invitations and replies directly to the
addressees, this is assumed to be false. In order to change the setting, add
the following parameter to the zimbra config.php and set the value to true.
This parameter replaces the old logic that made the decision based on the
client being Apple or not.
(Note: With SmartFolders this setting can be overridden per device - see below)
define('ZIMBRA_SERVER_INVITE_REPLY', true);
Added SmartFolders directive serverinvitereply
==============================================
Added SmartFolders directive serverinvitereply to allow overriding the
server ZIMBRA_SERVER_INVITE_REPLY setting on a per device basis.
In case the user wishes to override the server setting for sending Calendar
Invites/Replies from the server they can add a serverinvitereply directive.
The user can then turn on or off the setting for one or more devices by
specifying each deviceID and a true/false value
For example
*SyncConfig*
serverinvitereply&deviceid1=true&deviceid2=false
It is only necessary to list devices for which the user wishes to override the
server setting
Removed all references to $folderTypeInactive to simplify code
==============================================================
Removed all references to $folderTypeInactive to simplify code. Now set
folder include parameter to false for any folder whose active setting is
false.
Check existence of Message Fragment ['fr'] before setting preview
=================================================================
Check existence of Message Fragment ['fr'] before setting preview. If the
message has no body, it will have no preview Fragment either
Added check for Jetty error 'Error 503 Service Unavailable' to SoapRequest
==========================================================================
Added check for Jetty error 'Error 503 Service Unavailable' to SoapRequest.
This will help to identify installations that have zimbra 8 but have not
added the z-push IP address to the DoSFilter whitelist
For zimbra 8.0.3 or later, see http://wiki.zimbra.com/wiki/DoSFilter
For 8.0.0-8.0.2, see
http://www.zimbra.com/forums/announcements/60397-zcs-dosfilter-workaround-zcs-8-0-1-8-0-2-a.html
Fixed MeetingRequest organizer so email forwards/replies do not change it
=========================================================================
Fixed MeetingRequest organizer so email forwards/replies do not change it
In a reply to a meeting request, or a forward of the request, the organizer
was getting set to the From address. This has been fixed
Added throw FolderSync Exception if folder not found in index
=============================================================
Added throw FolderSync Exception if folder not found in index instead of
current situation where warnings will be generated for trying to get properties
of a non-object
Added error handling to file upload in SendMail
===============================================
Added error handling to file upload in SendMail. Checks are added to fopen, fwrite
and curl_exec to try to trap errors and return a false status to the device.
Fixed ChangesSink to detect folder changes in monitored linked folders
======================================================================
Fixed ChangesSink to detect folder changes in monitored linked folders. The detection
was only looking for notification on 'folder' - but some linked folders are botified
as 'link'
Added set $_userFolderTypeActive based on zimbra prefs in GetInfoRequest
========================================================================
Added set $_userFolderTypeActive based on zimbra prefs in GetInfoRequest. The default
values from the declaration will be overidden based on the "Features" available to the
user account. For example, if the attribute zimbraFeatureBriefcasesEnabled = "FALSE"
then $_userFolderTypeActive['document'] will be set to false.
Added Cleanup expired cachedMessageLists on Logoff
==================================================
Added Cleanup expired cachedMessageLists on Logoff. Gets rid of stale folder lists
from the cache file. Prior to this there was no way to remove a folder list from the
cache in the case of a folder deletion. This will now happen automatically after the
expiry time window (default 3600 seconds) has passed.
Fix QuickAdd yearly appointment - Make type 5 and extract month/dayofmonth
==========================================================================
Fix QuickAdd yearly appointment - Make type 5 and extract month/dayofmonth. If the
QuickAdd dialog is used to add a yearly appointment to the zimbra calendar, it gets
created on zimbra without the required fields for monthofyear and dayofmonth. Just
opening the appointment, clicking the customise on the recurrence, and clicking OK
and Saving the appointment adds those fields correctly. This fix looks for the
situation where the two required fields are absent, and extracts them from the
starttime of the appointment
Fixed DeleteMessage(appt) to call CancelAppointmentRequest if organizer
=======================================================================
Fixed DeleteMessage(appt) to call CancelAppointmentRequest if organizer. Calling
ItemActionRequest did not inform attendees of the meeting deletion/cancellation.
Calling CancelAppointmentRequest will send the requisite notifications. If the user
is not the organizer, then use ItemActionRequest as before
Fixed MoveMessage(appt) to call CancelAppointmentRequest if organizer
=====================================================================
Fixed MoveMessage(appt) to call CancelAppointmentRequest if organizer and the
appointment is being moved to the Trash. Calling ItemActionRequest did not inform
attendees of the meeting deletion/cancellation. Calling CancelAppointmentRequest
will send the requisite notifications. If the user is not the organizer, or the
destination folder for the Move is not the Trash, then use ItemActionRequest as
before
Added dynamic rate limiter _soapDelayMicroSeconds to SoapRequest
================================================================
Added dynamic rate limiter _soapDelayMicroSeconds to SoapRequest in order to try
to alleviate the DoSFilter issue for zimbra 8+ users. If the 503 Error page is
returned on a SOAP call, then the code will wait for one second, and add 40000
microseconds (1/25th of a second) to a delay setting that initially starts at 0.
This delay is used before every SOAP call for the remainder of the session.
If the 503 Error is returned again during the session - the steps will be repeated -
one second wait, and a further 40000 microseconds added to the inter-request delay.
The delay setting is reset to 0 for each new session. The long term fix for this
issue will remain to whitelist the server's IP address with the DoSFilter - but this
should alleviate the problem of generating empty versions of content records on the
device.
Change to not output the Briefcase folders in the Folder List
=============================================================
Change to not output the Briefcase folders in the Folder List. This is a fix for a
regression caused by adding the logic for "Sharepoint" Document access. z-push was
'fixing' the folder type to be type 18 (unknown) and sending it to the device. On
most devices this was ignored - but Outlook 2013 kept requesting a new FolderSync
to try to get rid of it. This resulted in an endless sync loop
Fixed to output recurring appointment exceptions(deletions) from CalDAV
=======================================================================
Fixed to output recurring appointment exceptions(deletions) from CalDAV. This issue
was reported on the zimbra forum. Deleting an instance of a recurring appointment
using CalDAV would remove the instance from the Web Client - but the exception was
not synching to the device. Found that zimbra CalDAV stores exceptions on calendar
records differently than the Web Client does. Change made to look for both.
Reorganized GetMessage (appointment) to minimise data send to device
====================================================================
Reorganized GetMessage (appointment) to minimise data send to device in the case of
deletion exceptions to Recurring Appointments. Just the startdate and deleted flag
are needed. Previously all fields were being sent to and then ignored by the device.
Trimming down the list to 2 fields to reduce bandwith
Added settings for _deviceMultiFolderSupport for Outlook and WindowsMail
========================================================================
Added settings for _deviceMultiFolderSupport for Outlook and WindowsMail. Both
appear to support multiple Calendar and Task folders. Neither supports multiple
Contact folders as far as I can tell. Neither has any Notes support.
Fixed interaction between _deviceMultiFolderSupport and Virtual folders
=======================================================================
Fixed interaction between _deviceMultiFolderSupport and Virtual folders. The
introduction of the _deviceMultiFolderSupport configurations had introduced some
unintended issues with the behaviour of Virtual folders, as in, if both Multi-
folders and Virtual folders were turned off for a folder type, the virtual
directive was ignored. Any combination of true/false for the two configuration
options should now work as intended.
Tidied up various log messages
==============================
Lots of tiding up of various log messages throughout the code
The following two fixes were added to the repackaged Release 57 dated 19-Nov-2013
Add check in SendMailSenderFix for full address as DisplayName
==============================================================
Add check in SendMailSenderFix for full address as DisplayName. This is added to
prevent "user@domain.com" <user@domain.com> from triggering the double-domain
check added above
Fix ZIMBRA_DISABLE_URL_OVERRIDE logic. If "true" do not override
================================================================
Fix ZIMBRA_DISABLE_URL_OVERRIDE logic. If "true" do not override. The logic in the
initial release 57 was backwards. Hopefully nobody had implemented the setting that
way.
Changes Made To Revision 56.1:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- In ChangeMessage for appointment - ensure contentParameters exists before
calling GetMessage()
- Check for existence of 'ms' and 'rev' fields in ModifyAppointmentResponse
as it seems zimbra 6.0.x does not report them
In ChangeMessage for appointment - ensure contentParameters exists
==================================================================
In ChangeMessage for appointment - ensure contentParameters exists before
calling GetMessage(). After publishing Release 56 it came to light that changing
an appointment on the device with z-push 2.0.7 would fail giving an error message like
the following
PHP Fatal error: Call to a member function GetContentClass() on a non-object
assuming your PHP was configured to show errors. This related to a difference between
version 2.1.x and 2.0.x of z-push - where the 2.0.x z-push versions were not setting
the contentParameters object before calling ChangeMessage in the backend. I have added
a check for the existence of the object, and if it is not found I create a default one
before continuing.
Check for existence of 'ms' and 'rev' fields in ModifyAppointmentResponse
=========================================================================
Check for existence of 'ms' and 'rev' fields in ModifyAppointmentResponse as it seems
zimbra 6.0.x does not report them
Changes Made To Revision 56:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Added function fixMS to handle json_decode "floated" millisecond timestamps
- Added handling for native Multiple Task folders for Apple devices - a new
switch has been defined for config.php - ZIMBRA_DISABLE_MULTI_TASK_LISTS -
see INSTALL for details
- Fix output of Exceptions to Recurring Appointments by using the new z-push-2
SyncAppointmentException class and fixing the exceptionstarttime
- Added handling of incoming Appointment Exceptions, including attendees
- Fix iPxxx "preview" of HTML-only email showing "this message has no content"
- Output Exception DTstamp instead of overall Appt DTstamp for exceptions
- Implemented ResolveRecipients - but fix needed in z-push-2 to allow this
to function correctly. This has been reported upstream.
- CALDAV created appointments do not seem to set OrganizerName and
OrganizerEmail. Check for the contents of both before building SOAP message
when modifying an appointment.
- Output lastverbexecuted - zimbra does not save sequence of verbs - just flags
so replied is given precedence over forwarded (same as GUI)
- Code cleanup - use $zimbraFolderId instead of overwriting $folderid
- Only output exceptions array on appointment if not empty
- After InitializePermanentStorage - check that a valid StateObject is returned
and if not, delay and re-read, to allow for limited file contention between
threads from the same device
- After InitializePermanentStorage - make sure _cachedMessageLists is an array
- Change in GetFolder to prevent fully shared (linked) accounts/folders from
being considered as special Mail folders (Inbox/Trash/Drafts/Sent)
- Fix in MeetingResponse to return empty string for meeting decline
- Added CustomRowCmp function in hope that z-push might adopt suggestion
Added function fixMS to handle json_decode "floated" millisecond timestamps
===========================================================================
The json_decode function does not handle millisecond timestamps on some
platform/php version combinations. This results in the timestamp getting
returned to the backend as a floating point number in scientific notation.
The fixMS function looks for numbers in this format and reformats them to
the expected 13 digit string
Added handling for native Multiple Task folders for Apple devices
=================================================================
Added handling for native Multiple Task folders for Apple devices - a new
switch has been defined for config.php - ZIMBRA_DISABLE_MULTI_TASK_LISTS -
see INSTALL for details
Fix output of Exceptions to Recurring Appointments
==================================================
Fix output of Exceptions to Recurring Appointments by using the new z-push-2
SyncAppointmentException class and fixing the exceptionstarttime
Added handling of incoming Appointment Exceptions
=================================================
Added handling of incoming Appointment Exceptions, including attendees
Fix iPxxx "preview" of HTML-only email showing "this message has no content"
============================================================================
Fix iPxxx "preview" of HTML-only email showing "this message has no content"
This would occur when a MIME message has html content only. It could also
result in the device not being able to download the message content
Output Exception DTstamp instead of overall Appt DTstamp for exceptions
=======================================================================
Output Exception DTstamp instead of overall Appt DTstamp for exceptions
Implemented ResolveRecipients - but fix needed in z-push-2
==========================================================
Implemented ResolveRecipients - but fix needed in z-push-2 to allow this
to function correctly. This has been reported upstream. Still awaiting
feedback on this one
CALDAV created appointments do not seem to set OrganizerName/OrganizerEmail
===========================================================================
CALDAV created appointments do not seem to set OrganizerName and
OrganizerEmail. Check for the contents of both before building SOAP message
when modifying an appointment.
Output lastverbexecuted
=======================
Output lastverbexecuted - zimbra does not save sequence of verbs - just flags
so replied is given precedence over forwarded (same as GUI)
Code cleanup - use $zimbraFolderId instead of overwriting $folderid
===================================================================
Code cleanup - use $zimbraFolderId instead of overwriting $folderid. This
makes it easier to follow the logic through different functions
Only output exceptions array on appointment if not empty
========================================================
Only output exceptions array on appointment if not empty. This avoids
generating warnings in the log
After InitializePermanentStorage - check that a valid StateObject is returned
=============================================================================
After InitializePermanentStorage - check that a valid StateObject is returned
and if not, delay and re-read, to allow for limited file contention between
threads from the same device.
If notifications are received simultaneously on two or more folders, there is
potential for the device to attempt to read the cache, clear the notified
folder, and save the cache near-simultaneously for two or more threads/folders.
This simple delay and re-read allows recovery of the cache in most cases.
After InitializePermanentStorage - make sure _cachedMessageLists is an array
============================================================================
After InitializePermanentStorage - make sure _cachedMessageLists is an array
If for any reason it is not a valid array then clear it and recreate it
Change in GetFolder for fully shared (linked) accounts/folders
==============================================================
Change in GetFolder to prevent fully shared (linked) accounts/folders from
being considered as special Mail folders (Inbox/Trash/Drafts/Sent). This
issue was causing the contents of the Inbox of a fully shared account to
appear merged into the users own Inbox
Fix in MeetingResponse to return empty string for meeting decline
=================================================================
Fix in MeetingResponse to return empty string for meeting decline.
The zimbra response for a meeting decline does not contain any meeting Id as
it does not create a meeting in this case. The lack of a value in the return
of calendarid to z-push was causing an error to be returned to the device.
Added CustomRowCmp function in hope that z-push might adopt suggestion
======================================================================
Added CustomRowCmp function in hope that z-push might adopt suggestion to
allow for custom sorting methods. The base comparison in function RowCmp in
diffstate.php uses
return $a["id"] < $b["id"] ? 1 : -1;
This is fine for all numeric-only IDs. But in zimbra, if a user shares their
folder with another user, the shared items get the other user's GUID prepended
to their IDs. This breaks the sorting, and results in loops and duplication of
contacts/tasks/appointments on any devices that use virtual folders.
My custom sort method for zimbra uses string comparison for that reason
return strcmp( $a["id"], $b["id"] ) < 0 ? 1 : -1;
Changes Made To Revision 55:
These changes are for the z-push-2 version ONLY
The z-push-1 and as12.1 versions are still at Revision 54
- Updated to match ConfigContentParameters changes in Z-Push 2.0.6 RC
(trunk change r1558) and later - Added $contentParameters=false to functions
MoveMessage, ChangeMessage, DeleteMessage and SetReadFlag. No
handling of $contentParameters has been added to the code at this time -
just the parameter has been added to the function definitions to avoid
breaking the backend
- Added zimbra/config.php file to match new packaging requirements of z-push
- Added unset()s for many foreach arrays/variables
- Moved ZIMBRA_USER_DIR handling into function GetUserProfileXML
- Moved FakeOutBox from function Setup() to function Logon()
- Differentiate default and user Appt/Contact folders in function GetFolder
- Changed ChangesSink to detect folder creation/deletion
- Changed ChangesSink to detect VirtualFolder changes
- Changed Log message prefix from ZimbraBackend to Zimbra for
consistency
- Fixed typos in CACHED TASKS/APPOINTMENTS log messages
Updated to match ConfigContentParameters changes in Z-Push 2.0.6 RC and later
=============================================================================
Added $contentParameters=false to functions MoveMessage, ChangeMessage,
DeleteMessage and SetReadFlag with a default value of false in case it is
called from a z-push version prior to Release 2.0.6RC. No additional handling of
$contentParameters has been added to the code at this time - just the parameter
has been added to the function definitions to avoid breaking the backend
Added zimbra/config.php file to match new packaging requirements of z-push
==========================================================================
To match the suggested new package layont for z-push backends, a zimbra/config.php
file has been added. The default file included will need to be updated to reflect
the zimbra settings from your specific install as contained in your existing
config.php - Then the zimbra specific settings should be removed from your z-push
config.php file.
Until this step is completed you will get warnings in the log file about duplicate
definitions
Added unset()s for many foreach arrays/variables
================================================
Cleaned up many arrays/variables immediately after use
Moved ZIMBRA_USER_DIR handling into function GetUserProfileXML
==============================================================
Moved ZIMBRA_USER_DIR handling into function GetUserProfileXML to avoid generating
unnecessary log messages for SmartFolder users
Moved FakeOutBox from function Setup() to function Logon()
==========================================================
Moved FakeOutBox from function Setup() to function Logon() where all others folders
are set up.
Differentiate default and user Appt/Contact folders in function GetFolder
=========================================================================
Differentiate default and user Appt/Contact folders in function GetFolder when
reporting the Folder Hierarchy to the device
For z-push-2 Changed ChangesSink to detect folder creation/deletion
===================================================================
Trigger a hierarchy resync if a folder is created/deleted
For z-push-2 Changed ChangesSink to detect VirtualFolder changes
================================================================
For z-push-2 Changed ChangesSink to detect VirtualFolder changes and request
a folder sync on the Primary folder for the appropriate item type
For z-push-2 Changed Log message prefix from ZimbraBackend to Zimbra
====================================================================
For z-push-2 Changed Log message prefix from ZimbraBackend to Zimbra for
consistency in some functions
Fixed typos in CACHED TASKS/APPOINTMENTS log messages
=====================================================
Fixed typos in CACHED TASKS/APPOINTMENTS log messages - a copy paste oversight
had them all reporting CACHED CONTACTS
Changes Made To Revision 54:
- Use session context refresh to populate _folders and _usertags during Logon
to minimise the number of SOAP requests for setup
- Change GetMpBodyRecursive to search for the part identified by zimbra as
containing the body text instead of piecing together body/inline text parts
- Change GetMessage to group Soap Request/Response into a single switch block
- Removed forcing parent to '0' for Calendars/Tasks
- Updated mime.php and mimePart.php in line with mail_mime 1.8.5 which contains
a fix for a bug we reported upstream concerning decoding headers
- Fixed messageclass for Meeting Replies/Cancellation
- Fixed timezone object creation issue that broke timezone matching on android
and resulted in DST changes moving recurring appointments
- Added sendasemail directive, and fixed regression in SendMail to force use of
_SendAsEmailOverride when configured by the user (directive or XML)
- Change SendMail to pass through encoded headers
- Fix regression regarding inline image only emails introduced in 53.7.z-push-2
- Added code to allow for not receiving a From: header from the (HTC) device
- Added code to encode From: header - if not encoded in _SendAsNameOverride
- Added basic check for incompatability between z-push and backend versions
- For z-push-2 fix truncation logic for BodyPreference[1] and [2]
- For z-push-2 BatchRequest plain and/or html bodies into a single SOAP Request
- For z-push-2 change to JSON for SOAP response processing - with the exception
of non SmartFolders folder initialization which uses a linear array to process
folders instead of a nested one. The MakeXMLTree function was mangling HTML
contained within XML nodes, and making it difficult to process.
- For z-push-2 remove Provisioning support code
- For z-push-2 remove AlterPing support code - replaced with ChangesSink
- For z-push-2 remove getSettings/setSettings - add Settings
- For z-push-2 report protocol version supported as 14 (ZPush::ASV_14;) even
though SMS sync is not supported - this version is needed for attendee
status updates to be shown on the device
- For z-push-2 rename function ItemOperationsEmptyFolderContent to EmptyFolder
- For z-push-2 fixed SendMail to check for content before setting parameters to
avoid "SyncMail: parameter 'reply_to' contains an invalid email address ''"
- For z-push-2 attachment->filereference has to be bin2hex encoded so that part
numbers for subparts/attachments do not lose their "." separators in the new
"evil" filters in request.php
- For z-push-2 add Utils:: prefix to call to function GetOLUidFromICalUid
- For z-push-2 fix notes handling for AS12+ appointments/tasks - keep original
notes if not updated (or update not sent - WM6.5)
- For z-push-2 add function GetUserInfo and move code from Logon into it to
ensure user profile info is known prior to calling GetUserProfileXML
- For z-push-2 Fixed extract of $id on contact creation to stop looping on
new contact created on the device.
- For z-push-2 refactored GetMessageList into a single switch block
- For z-push-2 added local caching of MessageLists to reduce server load on
initial sync where the same unchanged lists will be queried repeatedly
- For z-push-2 changed debugLog to ZLog::Write and adjusted logging levels for
errors in Logon so that obvious issues are highlighted without debug being on
- For z-push-2 added delay logic to ChangesSink in event of loss of connectivity
to the host
Use Session Context Refresh to populate _folders and _usertags
==============================================================
When issuing the AuthRequest command to logon to the backend it is possible to get
a session context refresh by specifying no session in the request. By using this
session context refresh to populate _folders and _usertags during Logon it minimises
the number of SOAP requests needed during the initial setup
Change GetMpBodyRecursive to look for zimbra highlighted Body part
==================================================================
Change GetMpBodyRecursive to search for the part identified by zimbra as
containing the body text instead of piecing together body/inline text parts. Note
inline text parts (such as disclaimers) still need to be appended
Change GetMessage to group Soap Request/Response into a single switch block
===========================================================================
A change of structure has been made to GetMessage to group Soap Request/Response
into a single switch block instead of one each for Request and Response sections.
This will make maintenance easier going forward
Removed forcing parent to '0' for Calendars/Tasks
=================================================
Historically the parent of Calendar/Task folders has been forced to Root ('0')
As zimbra 8 is due to allow nested calendars, this forcing of the parent to '0'
for Calendars/Tasks has been removed
Updated mime.php and mimePart.php in line with mail_mime 1.8.5
==============================================================
Updated mime.php and mimePart.php in line with mail_mime 1.8.5 which contains
a fix for a bug we reported upstream concerning decoding headers. Only change
from standard PEAR versions is removal of dependence on PEAR::isError
Fixed messageclass for Meeting Replies/Cancellation
===================================================
Following explanation of an issue with another backend on the z-push forum
a more granular (and correct) messageclass breakdown has been added
Fixed timezone object creation issue that broke timezone matching on android
============================================================================
Fixed timezone object creation issue that broke timezone matching on android
and resulted in DST changes moving recurring appointments.
Zimbra stores the day of the week starting at 1. However, the timezone object
needs the day to start at 0.
This difference broke the timezone matching algorithm on android and resulted
in timezones without daylight savings getting picked instead of those with it.
This would cause recurring appointments to jump an hour after the DST change.
Added sendasemail directive, and fixed regression in SendMail
=============================================================
Added sendasemail directive that can be used to change the sender's email
address when SmartFolders is configured. Note that if ZIMBRA_ENFORCE_VALID_EMAIL
is set to true - the configured email address will be overridden if is not a
valid zimbra alias for the user. Also, fixed regression in SendMail to force
use of _SendAsEmailOverride when configured by the user (directive or XML)
Change SendMail to pass through encoded headers
===============================================
Some phones pass through headers that are already encoded for 7-bit transport.
This change passes them through as they are rather than decoding them. This
should correct some problems with non-ascii characters in the To address,
Subject, etc.
Added code to allow for not receiving a From: header or receiving an empty one
==============================================================================
Added code to allow for not receiving a From: header or receiving an empty one.
Some devices do not include any From: header at all in the message they send,
while others have been found to send From: with no address after it. This fix
will look for both situations, and add a From: header using the the user's
preferred name/address
Added code to encode From: header - if not encoded in _SendAsNameOverride
=========================================================================
Added code to encode From: header. This is to allow non-ascii characters to
pass through in the sender's name. Note that this should now properly encode
the user's name from their zimbraDisplayName. If the user has specified an
override name through a SmartFolder directive or an XML file, this will be
checked to see if it is already in an encoded format - and will encode it
if it is not.
Added basic check for incompatability between z-push and backend versions
=========================================================================
Added basic check for incompatability between z-push and backend versions. As
a result of the number of questions opened by users who have installed z-push
2 without updating the zimbra backend - I have added a basic check that will
log to the debug log file if it detects that the backend is not compatable
with the z-push version.
For Z-PUSH-2
============
The following changes reflect the considerable architecture changes between
z-push 1.5 (and the as12.1 branch derived from it) and the new z-push-2
Fix regression regarding inline image only emails introduced in 53.7.z-push-2
=============================================================================
Fix regression regarding inline image only emails introduced in 53.7.z-push-2
Fix truncation logic for BodyPreference[1] and [2]
==================================================
Reworked body fetching and added truncation checks based on BodyPreference
BatchRequest plain and/or html bodies into a single SOAP Request
================================================================
If BodyPreference is supplied then request whichever body/bodies are
requested (1 and/or 2) in a single BatchRequest SOAP call
Change to JSON for SOAP response processing
===========================================
For z-push-2 change to JSON for SOAP response processing - with the exception
of non SmartFolders folder initialization which uses a linear array to process
folders instead of a nested one. The MakeXMLTree function was mangling HTML
contained within XML nodes, and making it difficult to process.
Remove Provisioning support code
================================
For z-push-2 remove Provisioning support code. As z-push-2 now has it's own
built-in privisioning support and no longer calls the functions previously
supplied by the backend - this code is being removed.
Remove AlterPing support code - replaced with ChangesSink
=========================================================
Though initially changed in Release 53 - this change is worth highlighting again
as it is significant, and to highlight the fact that a fix is available for the
previously mentioned bug in zimbra 7.2
For z-push-2 the old AlterPing method of detecting changes at the backend is no
longer supported so the related code is being removed. A new change detection
scheme, ChangesSink, has been introduced to replace it.
ChangesSink for zimbra depends on a SOAP call CreateWaitSetRequest which appears
to have deadlock issues in some versions of zimbra though I have not encountered
the problem during testing. See bug 74437 in the zimbra Bugzilla for more
information. This is fixed in 7.2 Patch 1, and the fix will be rolled into 7.2.1
I have no idea if this function has potential issues with zimbra Release 6.0.nn
For z-push-2 remove getSettings/setSettings - add Settings
==========================================================
For z-push-2 the getSettings/setSettings backend functions that were added for the
as12.1 branch support are not used so the code has been removed. z-push-2 instead
uses a single backend function Settings that combines the functionality of both
For z-push-2 report protocol version supported as 14 (ZPush::ASV_14;)
=====================================================================
For z-push-2 report protocol version supported as 14 (ZPush::ASV_14;) even
though SMS sync is not supported - this version is needed for attendee
status updates to be shown on the device
There is ongoing work on the z-push-2 server side to handle calls to sync SMS even
if SMS sync is not available in the backend.
For the moment though it is best to UNCHECK [ ] Sync SMS/Text Messages on your
device when setting up sync.
For z-push-2 rename function ItemOperationsEmptyFolderContent to EmptyFolder
============================================================================
For z-push-2 rename function ItemOperationsEmptyFolderContent to EmptyFolder
This function was previously added to support the as12.1 branch though the hooks
were never added by the branch developer to call the function. z-push-2 uses the
new name for the function
For z-push-2 fixed SendMail to check for content before setting parameters
==========================================================================
For z-push-2 fixed SendMail to check for content before setting parameters to
avoid "SyncMail: parameter 'reply_to' contains an invalid email address ''"
z-push 1.5 used to handle these settings silently so it was never an issue. With
z-push-2 each instance is flagged with log entries so pre-screening has been added
For z-push-2 attachment->filereference has to be bin2hex encoded
================================================================
For z-push-2 attachment->filereference has to be bin2hex encoded so that part
numbers for subparts/attachments do not lose their "." separators in the new
"evil" filters in request.php
Previously the attachment filereference could be sent to the device in plain text.
For example, 2:12345:2.1 represented folder 2, message 12345, MIME part 2.1
The new z-push-2 evil content filter strips out the "." character which left any
device requesting part 2.1 of the message generating a backend lookup for part 21
instead. This would either fail completely or in the unlikely event that there
were 21 parts to the MIME message would return the incorrect part to the device.
bin2hex encoding the filereference and then hex2bin decoding it whenever a request
is made to retrieve it was suggested by the z-push-2 developers as the way to work
around this issue.
For z-push-2 add Utils:: prefix to call to function GetOLUidFromICalUid
=======================================================================
Having eliminated the dependency on a local copy of the as12.1 utils.php file, this
call needs a prefix to identify the function as being in the z-push-2 Utils class
For z-push-2 fix notes handling for AS12+ appointments/tasks
============================================================
For z-push-2 fix notes handling for AS12+ appointments/tasks - keep original
notes if not updated - this fix is as a result of WM6.5 not sending over any
updates to the description fiels for appointments/tasks
For z-push-2 add function GetUserInfo and move code from Logon
==============================================================
For z-push-2 add function GetUserInfo and move code from Logon into it to
ensure user profile info is known prior to calling GetUserProfileXML.
The refactoring of the logon and setup routines had broken the processing
of user's email addresses for those using XML files.
For z-push-2 Fixed extract of $id on contact creation to stop looping
=====================================================================
For z-push-2 Fixed extract of $id on contact creation to stop looping on
new contact created on the device. The code to extract the newly created
server contact id was not correct, so it was returning false to the device
which was then retrying to send the new contact over. This loop would
duplicate the contact on the server over and over
For z-push-2 refactored GetMessageList into a single switch block
=================================================================
For z-push-2 refactored GetMessageList into a single switch block in order
to simplify the implementation of local cacheing
For z-push-2 added local caching of MessageLists to reduce server load
======================================================================
For z-push-2 added local caching of MessageLists to reduce server load on
initial sync where the same unchanged lists will be queried repeatedly
within a short space of time. The default is for this feature to be enabled
with each folder cache having a lifetime of 3600 seconds.
This feature is enabled by default - but can be disabled by adding a new
config.php directive
define('ZIMBRA_LOCAL_CACHE', false);
The cache for each folder has a default lifetime of 3600 seconds. This can
be adjusted by adding an additional directive
define('ZIMBRA_LOCAL_CACHE_LIFETIME', 300);
For z-push-2 changed debugLog to ZLog::Write and adjusted logging levels
========================================================================
For z-push-2 changed debugLog to ZLog::Write and adjusted logging levels for
errors in Logon so that obvious issues are highlighted without debug being on.
For example, if php-curl is not installed OR if the ZIMBRA_URL is not defined
the sync cannot work. These will now be logged at a FATAL level. Other entries
have been set at INFO level so they get logged on a default install.
For z-push-2 added delay logic to ChangesSink in event of loss of connectivity
==============================================================================
For z-push-2 added delay logic to ChangesSink in event of loss of connectivity
to the host. This is to prevent the code from looping trying to create a
WaitSet. If the ChangesSink was supposed to be for 60 seconds, and the CURL
return shows a problem connecting to the backend, the code will simply delay
for the 60 seconds before returning control back to z-push
Changes Made To Revision 53:
- Implemented ChangeFolder and DeleteFolder
- Don't send invitations/updates for meetings created/modified on Apple iPxxx
devices, as device is sending them directly
- Fixed set 'active' settings for document/wiki to false in GetUserProfileXML
- Fixed debug log error in ReportMemory and report "Unavailable" when function
to return memory usage is not found
- Mail folders (at least) on servers upgraded from earlier versions of zimbra
do not have the view property set - default it to "message"
- Update SmartFolder handling to use defaulted "view" for efficiency
- Added support for storing lastsyncip in provisioning table
- Removed domain portion of userid (if supplied) in provisioning functions
- Move creation of SyncMeetingRequest object up to avoid "Creating default
object from empty value (2048)" error
- Split bodyPreference Wanted message over two debugLog lines to avoid
undefined index error in debug log
- Added missing truncation choices in GetTruncsize function - Defined constant
values not used so as not to deoend on zpushdefs.php getting updated
- Fix for sender email address corruption when ZIMBRA_ENFORCE_VALID_EMAIL is
not defined - or not true
- Removed subtype from debug line in GetAllBodyRecursive
- Added back contentlocation for Attachments and revert change to attoid as
attachments were not opening correctly or were disappearing from the emails
on the device
- Fixed SendMail for z-push-2 - Use SyncSendMail object as only parameter
- For z-push-2 - Use WaitSet's for triggering push
- For z-push-2 - bump limit in GetMessageList to 500 from 100
- For z-push-2 - Change GetMailboxSearchResults to work with CPO object
Implemented ChangeFolder and DeleteFolder
=========================================
Implemented the server operations ChangeFolder and DeleteFolder
Don't send invitations/updates for meetings created/modified on Apple iPxxx
===========================================================================
Apple iPxxx devices send meeting invitations directly from the device. For
all normal meeting invitations/updates zimbra will send emails to all invitees
automatically on saving the meeting. A change has been added to detect the
UserAgent of the device creating/changing the meeting, and if it is an Apple
device to bypass the sending of the emails from Zimbra. This stops the
duplication of invites/updates.
There is a known side effect though that the sender name cannot be updated
as it normally would - so the email will usually have only an email address as
the From: person - not "Joe Sender" joe.sender@wherever.com
Unfortunately the Meeting Acceptance API does not have the same switch
available to eliminate sending duplicate acceptances/rejections of meetings
from Apple devices.
Fixed set 'active' settings for document/wiki to false in GetUserProfileXML
===========================================================================
For previously upgraded zimbra systems, there can still be folder types of
document and wiki. There was no default set for these folder types in the
User XML handling code. This has been added.
Fixed debug log error in ReportMemory and report "Unavailable"
==============================================================
A fix was added to report "Unavailable" rather than an error message if the
PHP extension to gather memory statistics is not available on the server.
Set default view property of folders to "message"
=================================================
Mail folders (at least) on servers upgraded from earlier versions of zimbra
do not have the view property set - default it to "message" if it does not
already have a view property.
Update SmartFolder handling to use defaulted "view" for efficiency
==================================================================
The SmartFolder handling code was able to be streamlined as a result of
knowing that all folders would have a "view" property - a knock on benefit
from the previously mentioned fix.
Added support for storing lastsyncip in provisioning table
==========================================================
For z-push 1.5.x and as12.1 branches, the zimbra backend provisioning functions
can be enabled. One user requested the addition of the last synced IP Address
to the provisioning table. This was added to the backend - but not as yet to the
provisioning GUI.
Removed domain portion of userid (if supplied) in provisioning functions
========================================================================
In the case where a full DOMAIN\username is entered on the device, the presense
of the "\" was seen by one user to cause problems for the provisioning GUI pages.
A change was made to only pass the username to the backend database.
Move creation of SyncMeetingRequest object up
=============================================
The creation of SyncMeetingRequest object was moved up to avoid "Creating
default object from empty value (2048)" error
Split bodyPreference Wanted message over two debugLog lines
===========================================================
For as12.1 the backend could receive a bodyPreference Wanted indicator - or not
depending on the device synching. If this was not sent to the backend an
undefined index error in debug log. This debug has been split over 2 lines so
as to check for the presence of the wanted flag prior to logging what it is set
to.
Added missing truncation choices in GetTruncsize function
=========================================================
The GetTruncation function was not handling all possible values being sent from
the device. A fix has been added to support the missing values. Note that hard-
coded values are checked rather than defined constant values - so as to make sure
the fix is not dependant on zpushdefs.php getting updated. A bug report on the
base problem has been raised with z-push.
Fix for sender email address corruption
=======================================
A fix was added to prevent the From: address getting corrupted when
ZIMBRA_ENFORCE_VALID_EMAIL is not defined - or not true.
Removed subtype from debug line in GetAllBodyRecursive
======================================================
The subtype is no longer passed to GetAllBodyRecursive - so when the
ZIMBRA_DEBUG flag was set, the log entry for the function would write an error
to the debug.txt file.
Added back contentlocation for Attachments and revert change to attoid
======================================================================
A change made in Release 52 to support the sending of images from iOS5 devices
caused an unexpected problem with som other attachments. This would manifest itself
as an email in the devices list appearing to have attachments - but when clicked on
they would not download. On closing the email and going back into it the attachments
would no longer show as being available.
A change was made to add back the contentlocation property for Attachments and
revert change to attoid as attachments were not opening correctly or were
disappearing from the emails on the device.
There may still be some issues with some attachments - but this should behave a
great deal better than Releae 52.
For Z-PUSH-2
Fixed SendMail for z-push-2 - Use SyncSendMail object as only parameter
=======================================================================
Z-PUSH-2 has changed the function definition of the SendMail function. A change
has been made to bring the z-push-2 version of the backend into line with the
now current function definitions.
Use WaitSet's for triggering push
=================================
Z-PUSH-2 no longer uses the AlterPing methodology for detecting changes at the
server. A new ChangesSink methodology has been introduced instead. The backend
has been updated to use zimbra WaitSets - which should reduce the load on the
zimbra server. Note that zimbra 7.2.0 has a bug in WaitSets that is due to be
fixed in 7.2.1
Bump limit in GetMessageList to 500 from 100
============================================
To reduce the number of backend hits for large mailboxes - the maximum number
of rows that can be handled per backend request has been increased from 100 to
500 in GetMessageList
Change GetMailboxSearchResults to work with CPO object
======================================================
Z-PUSH-2 has changed the function definition of GetMailboxSearchResults to only
use a single CPO object. A change has been made to bring the z-push-2 version
of the backend into line with the now current function definitions.
Changes Made To Revision 52:
- Fixed length encoding in GlobalObjID that caused duplicated meetings, and
broke meeting updates - Fix can't update bad meeting IDs already on device
- Fix to use client generated UID for new appointments created on the phone
to avoid duplication of the appointment in the calendar
- Fix to strip Email Name from email address on contact sync - android issue
- Changed handling of virtual folders for Apple devices as they have a non-
standard ability to display multiple Calendars and Multiple folders of
contacts natively. This change will allow all Calendar and Contact folders
to be synced to Apple devices. I have tested WM 6.5, Samsung android 2.3.3
and Nokia, and none of them can support this functionality. If anyone knows
other devices that have this native capability, let me know and I can add
exceptions to the code for them as well. To turn off this feature two new
switches have been defined for config.php - ZIMBRA_DISABLE_MULTI_CALENDARS
and ZIMBRA_DISABLE_MULTI_CONTACT_GROUPS - see below for details
- Fixed Plain Text attachments getting appended to plain text body
- Fixed Attachments/Images for MIME formatted messages
- Added HTML/MIME formatted message support for iPhone using z-push 1.5.x -
Setting the ZIMBRA_HTML flag to true should now send MIME to the device when
requested by the device (mimesupport 2 requested).
- Fixed S/MIME Support. S/MIME messages now sent to device when mimesupport 1
received from device.
- Changed meetingstatus to 1 (Is a meeting) from 0 for appointment sync
- Added new optional config.php setting ZIMBRA_MB_DETECT_ORDER to allow
overriding of the default list of charsets. See notes below.
- Change to GetHTMLAttachmentsRecursive to allow for content-type header that
has no primary/secondary type included
- Added 86400 second offset to message $cutoffdate as zimbra search works
from midnight and just using $cutoffdate turned into a date string was
dropping all emails from 3rd day back. Now seach returns today, and all of
previous 3 days/7 days/14 days etc.
- Changed $revision to $GLOBALS['revision']
- Added optional configuration ZIMBRA_DETECT_RUSSIAN to allow support of the
Russian language. Requires include file a.charset.php
- Fix for Modify Appointment function ConvertItemIdToInvID to use
GetAppointmentRequest instead of SearchRequest which was sometimes unable
to find all appointments
- Fix to only send InstanceType for meeting Request - not for Notification
- Fix for GetAttachmentData and ItemOperationsGetAttachmentData to check if the
attachment is in a Shared Mailbox and handle $attname differently.
- Fix for GetAttachmentData to not output a content-type header, as request.php
is already sending a generic application/octet-stream content type header.
Also added output Content-Length header, and use echo instead of print
- Fix added to GetMpBodyRecursive for handling inline images from iOS5
- Replaced GetBodyRecursive with GetAllBodyRecursive to streamline processing
- Uncomment addHTMLImage call in getHTMLAttachmentsRecursive
- Fix added to SendMail to handle attachments on SmartForward requests
- Fix added to StatMessage to detect Shared Contacts
- Fix added to GetMessage to search for the folderid if not supplied
- Added handling of new *SyncConfig* directive "sendasname" to override the
name retrieved from zimbra (see below for details)
- Removed hard limit of 1MB from Fetch function
- Added instructions for selected user debugging with ZIMBRA_DEBUG flag
- Added function ItemOperationsEmptyFolderContent for as12.1 branch to
enable 'Empty server trash' command from the device
- Added function ResolveRecipients for as12.1 branch
- Updated Search function to allow for a subset of search parameters being
sent over from the device for as12.1 branch
In addition - for z-push-2 alpha - the following additional changes have been made
- First port of backend to z-push-2 architecture
- Added _deviceAgent, and changed code to work with _deviceAgent instead of
$GLOBALS['useragent'] for z-push-2
- Moved Search functions into a separate class for z-push-2
- Added function GetGALSearchResults for z-push-2
GlobalObjID Encoded Length Fix
==============================
In Release 51 a change was added to try to address calendar item duplication
and the failure of the device to recognise updates to calendar items on some
devices. That fix had an error in the way it encoded the UID length within
the GlobalObjID. This has been corrected.
Use Client Generated UID
========================
When new meeting requests are sent from the phone, it generates a UID for the
meeting. Zimbra will allow the use of a client generated UID in a new meeting
creation request, so a change has been added to now use the UID from the phone.
This should finally stop the duplication of meetings.
Strip Display Name from Email Address
=====================================
When a contact is changed on an Android phone (and possibly others) it sends the
email address back in the format "Full Name" <account@domain.com> - this causes
problems on zimbra. A fix has been added to strip off the name.
Multi-Folder Support Added (for Apple iOS)
==========================================
Apple devices have the ability to handle multiple calendars and contact folders
natively. If an Apple device is detected, the additional coneacts and calendar
will not be "virtually included" into the default folders Contacts and Calendar
but will instead be sent across in their own folders.
If anyone is aware of any other devices that support this capability, please
post a message on the Support Requests Tracker so we can add a check for that
device to the code.
To Disable the breaking out of Calendars and Contact Groups on devices
----------------------------------------------------------------------
By default, if an Apple device is identified, Calendars and Contacts in custom
folders will be passed to the device in separate calendars/folders. This allows the
device user to select what they want to display. It potentially increases the load
on the backend to have to sync a lot of folders though. In order to prevent devices
synching many different folders, the administrator can disable the support of
multiple calendars and/or multiple contact folder using these two switches.
define('ZIMBRA_DISABLE_MULTI_CALENDARS',true);
define('ZIMBRA_DISABLE_MULTI_CONTACT_GROUPS',true);
Note that if either feature is disabled, the behaviour of the backend will return to
that which was in place before the breakout ability was added. i.e. all items will get
virtually included in the primary folder (assuming virtual support is turned on)
Fix Plain Text Attachments
==========================
A fix has been added to stop text/plain attachments from being appended to the
plain text body of an email. They should now appear correctly as attachments.
MIME Image/Attachment Now Handled
=================================
The handling of Images and Attachments in a MIME formatted message has been
re-worked, and should hopefully now work correctly.
HTML Email for IPhone using z-push 1.5.x
========================================
Support for HTML (actually MIME) email for iOS devices has been added. This is
controlled by the ZIMBRA_HTML flag which has never worked correctly up until now.
If your device requests MIME formatted email, and the ZIMBRA_HTML flag is set to
true, then we will now send MIME to the device. Thsi will allow the device to
display full HTML email.
S/MIME Support
==============
S/MIME messages will now be sent to device unaltered when a mimesupport 1
request is received from device.
MeetingStatus Flag Changed
==========================
The meetingstatus flag on appointments was changed from 0 (unknown) to 1 (meeting)
MB Encoding Detection Extended by 24hrs
=======================================
A Japanese user had an issue with the character set encoding detection that was
causing them to get garbled emails. After troubleshooting the issue it was
identified that a more flexible method of supplying character sets to the
decoding algorithm was needed.
A new optional config.php setting ZIMBRA_MB_DETECT_ORDER has been added to allow
overriding of the default list of charsets ('ASCII, UTF-8, ISO-8859-1, ISO-8859-15')
If you wish to use a different set of charsets - or to change the detection order
then simply add a new definition to the config.php file - for example - the Japanese
user used the following override
define('ZIMBRA_MB_DETECT_ORDER','ASCII, ISO-2022-JP, UTF-8, ISO-8859-1, ISO-8859-15');
which correctly detected the Japanese encoding of emails for them.
See PHP documentation for mb_detect_order for details. Be careful of the sequence of
charsets - as the detection will stop as soon as it finds a possible match - even if
a later charset in the list is the correct match.
Note: I am still working with a Russian user looking for a workaround for the fact
that the inbuilt PHP functions do not detect cyrillic encodings.
Attachments with no primary/secondary content-type
==================================================
Some phones (Cyanogenmod android) add attachments with a broken content-type header.
The code was dropping these attachments. A fix has been added to check for this
condition, and to add the attachment as unknown type.
Message/Calendar Filter Windows Extended
========================================
When a user selects for example - email for last 3 days - on their phone, z-push
uses a rolling 3-day (or 72 hour to be exact) window to determine what messages to
send to the phone. The filtering on zimbra behaves slightly differently, however,
and it would filter based on the day - not the actual time of day. So, on the phone
you would always get today, yesterday and the previous day. You would not get the
messages from the messages from the third day back that were still within the 72-hour
window. A change has been made to err on the side of showing too many rather than too
few messages - so the code will now add an extra day of messages to all selection
criteria. So, now the seach returns today, and all of previous 3 days/7 days/14 days,
etc.
Use $GLOBALS['revision'] instead of $revision
=============================================
Some systems were not properly reporting the revision in the logs. Adding the $GLOBALS
wrapper addresses this.
Added option to handle Russian encoding
=======================================
The normal mbstring functions cannot identify Russian encodings - as they are not a
character set - but rather an encoding. The default behaviour of the zimbra backend
mangles the Russian characters as a result of this. Working with a russian user we
have found a reliable php script on the internet that can identify the different
Russian encodings, and convert the text into Windows-1251 which the zimbra backend
knows how to handle. If you wish to use this functionality, you will need to download
the file a.charset.php from the SVN (misc folder) and place it in the include folder
on your z-push installation. Then add the option key
define('ZIMBRA_DETECT_RUSSIAN', true);
to the config.php file.
Fix for Modify Appointment
==========================
In some instances the SearchRequest function would not find a Calendar appointment
so when the user tried to change an appointment from their device, the modification
would fail. A fix has been made in the function ConvertItemIdToInvID to use the SOAP
call GetAppointmentRequest instead as this appears to be a more reliable method to
get the appointment Id.
Fix for Meeting Notification - Remove InstanceType
==================================================
Fix to only send InstanceType for a meeting Request. Previously it was also being
sent for Meeting Notification messages as well.
Fix for GetAttachmentData and ItemOperationsGetAttachmentData - Shared Mailbox
==============================================================================
A fix was added to GetAttachmentData and ItemOperationsGetAttachmentData to
check if the email is in a shared mail folder and handle it appropriately. The
supplied $attname is normally a 3-part identifier - FolderId:MessageId:MessagePart -
however, in the case of a shared mail folder, the MessageId element itself contains
a : separator between the userid of the user who shared the mail folder, and the
MessageId for that user.
Fix for GetAttachmentData to not output a content-type header
=============================================================
Fix for GetAttachmentData to not output a content-type header, as request.php
is already sending a generic application/octet-stream content type header.
Also added output Content-Length header, and use echo instead of print
Fix added to GetMpBodyRecursive for handling inline images
==========================================================
Fix added to GetMpBodyRecursive for handling inline images contained in emails
sent from iOS5 primarily. The images are contained in MIME part identified as an
inline image - but the image is not referenced anywhere within the body of the
email. The parent MIME type is multipart/mixed.
The fix, adds a placeholder for the image if it is building an HTML mail body and
adds the image as an attachment. The resulting parent container is of type
multipart/related.
This may be re-worked again in a future release if a better way to handle it can
be found.
Replaced GetBodyRecursive with GetAllBodyRecursive to streamline processing
===========================================================================
Multiple calls to GetBodyRecursive have been replaced with a single call to a
replacement function GetAllBodyRecursive. This allows for better handling of
multipart messages - particularly those containing inline files
Uncomment addHTMLImage call in getHTMLAttachmentsRecursive
==========================================================
The call to addHTMLImage in getHTMLAttachmentsRecursive has been uncommented as
it was found to be necessary in order to get all inline images to get sent to
the device
Fix added to SendMail to handle attachments on SmartForward requests
====================================================================
A fix has been added to SendMail to handle attachments on SmartForward requests.
SmartReply requests will not include the attachments - so the user should forward
the email back to the original sender and add other recipients - if they wish to
pass on the original email with comments added, and with the attachments included
Fix added to StatMessage to detect Shared Contacts
==================================================
A fix has been added to StatMessage to detect Shared Contacts and to handle them
appropriately. The ownerId of the user who shared the contact needs to be included
in the item Id for any contact update request.
Fix added to GetMessage to search for the folderid if not supplied
==================================================================
Some GetMessage requests were being received with no folderId included. In the
absense of a folderId, it is not possible to identify the type of item we are
processing - i.e. message/contact/appt/task. A fix has been added to GetMessage
to search for the folderid using the item Id if the folder Id is not supplied
Removed hard limit of 1MB from Fetch function
=============================================
An arbitrary limit of 1MB has been removed from the Fetch function to avoid
breaking the fetch for large attachments
Added instructions for selected user debugging with ZIMBRA_DEBUG flag
=====================================================================
Added instructions to the notes at the top of the zimbra.php file for selected user
debugging with ZIMBRA_DEBUG flag - the feature was added back in release 48 and was
documented in the release notes as follows
- Added option to define ZIMBRA_DEBUG as 'user1' or 'user1,user2,userN' to
enable zimbra soap debug messages for just the listed accounts. Defining
it as true/false will continue to work as before
Added function ItemOperationsEmptyFolderContent for as12.1 branch
=================================================================
Added function ItemOperationsEmptyFolderContent for as12.1 branch to enable
'Empty server trash' command from the device. This is used where a user who is
remote from their office has exceeded their mailbox quota, and they need to free
up some space to allow emails to come into their mailbox. The functionality will
not work until the z-push as12.1 branch adds a handler to call this code. I have
requested this from the developer.
Added function ResolveRecipients for as12.1 branch
==================================================
Added function ResolveRecipients for as12.1 branch. This function verifies email
addresses against the backend
Updated Search function for compatability with newer devices
============================================================
Updated Search function to allow for a subset of search parameters being sent
over from the device for as12.1 branch. For example the Samsung phones running
android 2.3.6 can send start/end date ranges, folder id, etc.
New *SyncConfig* directive added - sendasname
=============================================
Added handling of new *SyncConfig* directive "sendasname" to override the
dispalay name retrieved from zimbra. This was primarily added to counteract the
corruption of the sender name in emails where non-ascii characters are included
in the sender's name - To counteract this the sendasname value can be set to what
it would be if correctly MIME-encoded.
To get the correctly formatted name, send yourself an email through the web
client, and then right-click on the received email and Show Original. Look for
the From: header and copy the name from there. Don't include the email address
part - or any surrounding "" marks. Note that the encoding zimbra uses will most
likely be utf-8 rather than ISO-8859-1 as seen in the following example
For example
=?ISO-8859-1?Q?Andr=E9?= Pirard
The example above is taken from the rfc2047 document, and appears in a CC:
CC: =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>
For reference: http://www.faqs.org/rfcs/rfc2047.html
z-push-2 - alpha release
========================
In addition - for z-push-2 alpha - the following additional changes have been made.
This code is still in early testing as is z-push-2 and will need to change in line
with the constant changes being made to z-push-2 at the moment. Hence it is not
being released in svn at this time.
- First port of backend to z-push-2 architecture
- Added _deviceAgent, and changed code to work with _deviceAgent instead of
$GLOBALS['useragent'] for z-push-2
- Moved Search functions into a separate class for z-push-2
- Added function GetGALSearchResults for z-push-2
Changes Made To Revision 51:
- Added attendee status/type information to appointments
- Changed GlobalObjID to be correct non-outlook encoded value so that Apple
and possibly other devices will update appointments correctly
- Changed handling of invitations for client generated meetings - zimbra will
always send out invitations now (even for Apple/Android) - but if zimbra is
the default email client on the device, we will drop the device originated
invitation once it is received by SendMail. If another email account is your
default account then the device will send a "duplicate" meeting invitation
directly through that email account. Replies to the device originated
invitation may not correctly update the attendee status of the appointment
on the original device
- Fixed as12+ body handling for Appointment/Contact/Task
- Changed handling of birthday/anniversary field to try to get a solution
that will work for most scenarios
- android 2.3.3 seems to have a serious problem with the birthday field that
causes it to spin constantly trying to resync the calendar - limiting
birthday to known UserAgents for now (Apple and MailforExchange)
- Experimental fix in Logon to delay and retry logon attempts when the
zimbra server is not available (eg. zimbra FOSS shut down for backup)
See ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR in the notes below.
- Fix to add missing $this-> prefix to ScrubHtmlText in function
GetMpBodyRecursive
- Added fix to AlterPingChanges to detect changes in Virtual Folders
- Added output the Conversation ID field to allow view by conversation
on some Android devices (WM appears to need more to work)
- Cosmetic fix in GetSearchResults to avoid logging error when GAL search
returns zero matches
- Added 'wiki' folder type for SmartFolder setup to accomodate servers that
are running, or once ran, version 5.0.x of zimbra
- Removed Nokia only filter on adding FakeOutbox folder, as Android 2.3.3
seems to need it too, and it doesn't appear to break other phone syncs
and renamed the devid to FakeOutbox from NokiaFakeOut
- Added path to fake outbox structure to fix debug log error
- Save SOAP error extracted from response to $this->_soapError for
decision making in calling routine
- Removed 'Too big for inline?' debug message
- Reworked Sendmail address checking
- Revised non-plain text email handling in GetMessage
- Reverted text/html encoding to base64 from quoted-printable for mime body
as the majority of incoming messages use base64
Attendee Status & Type
======================
The status (Accepted/Tentative/Declined/Unknown) and type (Required/Optional/Resource)
information for meeting attendees are now pushed out to the device.
Changed Global Object ID handling
=================================
In some cases, zimbra meeting updates were not properly updating the client device.
Changed the handling of the GlobalObjId field to encode it as a "non-Outlook" ID to
improve the update reliability
Meeting Requests and Duplicate Invites (Apple and Android??)
============================================================
Having struggled to understand the behaviour of the Apple iP*** devices and their
handling of meeting requests, I was finally able to get access to an iPod (4.3) to
test with.
What I discovered is that whenever you set up a meeting on an Apple device, the
device will create the meeting on the server, and once it gets confirmation that the
meeting was created successfully, it will then send out emails directly from the
device to the invitees in a 3-part MIME message inviting them to the meeting. The
third part of the MIME message is of type text/calendar and contains an ICS meeting
request. Zimbra by default also sends out meeting requests - so the recipient will
end up receiving 2 separate invites to the same meeting.
I also found that if I accepted the meeting request sent from the device directly,
it did not update properly update the meeting attendee status for the meeting.
For these reason, I have decided to adopt the following processing rules.
1. zimbra will be allowed to invite participants to all meetings
2. if the device-originated invitations are sent through zimbra - I will drop them
without actually sending the email to the recipient.
This apporach gives the best outcome in terms of allowing for correct handling of
replies to the meeting request, and status updates, etc.
There is still no way to intercept the duplicate meeting invitation from the device
however, if it is configured with multiple email accounts, and it routes the
direct invitation out through one of the alternatie accounts. I believe you can
select the "Default Account" in the Email settings on the Apple devices to force
email to go through the ActiveSync account as a preference - which would allow all
duplicate invites to be caught and dropped.
Fix AS12+ Body Handling
=======================
Just as AS12+ brought with it the ability to specify the preferred content of emails
it also brought the same ability to Appointment/Contact/Task body fields. This release
fixes the handling of the body field when an AS12+ server is being used.
Changed handling of Birthday/Anniversary
========================================
In troubleshooting a birthday field issue raised by an iPhone user, I found the
original scheme employed in the code to try to ensure the date conversion would always
work was causing issues with the iPhone. Whatever date the user set on zimbra would
get incremented by one day on the iPhone. After some experimentation I think I have
found an alternative thet will work more reliably with most if not all phones. This
has been implemented. (See also Birthday Field Breaks Android 2.3.3 Sync below)
Birthday Field Breaks Android 2.3.3 Sync
========================================
Having done extensive troubleshooting over a number of months on an issue where my
own phone, having been updated to android 2.3.3, began to constantly sync the
calendar regardless of the ActiveSync "ping" settings, I finally stumbled on the
solution to the problem while working on the iPhone Brithday issue above.
It seems that Android 2.3.3 does not like the birthday field as presented by z-push
/zimbra. I have no way to know if a real Microsoft Exchange Server birthday field will
behave the same way - but I assume it will. If the birthday field is present on the
server, the Calendar on the phone will constantly sync - connecting every minute -
which will run up data charges and drain the battery rapidly on the device. I assume
the calendar is trying to look for, or create a "birthday" appointment with which to
sync - but nothing happens other than the constant spinning.
For this reason, I have limited the devices that the birthday field will sync to to
those devices I know do not have such problems. Nokia (MailforExchange) and Apple
are the only unique UserAgent strings I have been able to allow to work for now.
For any other device, the contact record will still sync, just the birthday field
will not be sent to the device.
Experimental fix in Logon to delay and retry logon
==================================================
Like many Zimbra FOSS users, we use a backup script from the Administrators forum
that requires zimbra to be shutdown briefly to do a cold-sync backup. For some
mobile devices, if they try to initiate a connection to the server during that
period, they will spin quickly retrying and will then give-up. Other devices try
again after a period of time has elapsed.
This experimental fix has been put in place to allow a few minutes for the server
to become available again so that the device does not get into this "I've given up"
state. It has been working well for me.
To enable the functionality ass a new definition in the config.php file
define('ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR',5);
and set the value appropriately for your system.
I am not aware of the tolerance of phones towards communication delays - hence the
experimental label.
Fix for ScrubHtmlText
=====================
A fix has been implemented to add missing $this-> prefix to ScrubHtmlText in function
GetMpBodyRecursive. This bug was causing the script to terminate in the event that
the phone requested text output, but the email was in HTML only.
Fix for "Ping"ing Virtual Folders
=================================
A fix was added to AlterPingChanges to detect changes in Virtual Folders. These
folders are treated as extensions of the main Calendar/Tasks/Contacts folders,
and their content is synched over to the equivalent folders on the phone. However,
an issue was identified whereby a change to an item in the virtual folder would not
trigger a resync of the folder. This has been fixed.
A word of warning for anyone who shares folders among very large groups of users is
that all users would get the notification to sync within the same 5 second time
window - and this could add extra load spikes to your Apache and/or Zimbra servers.
Conversation View (partial support)
===================================
While zimbra does not support the full conversation view history that Exchange does,
adding the Conversation ID at least allows some phones to group emails together
correctly by Conversation. This has been tested under the as12.1 configuration
using an android 2.3.3 phone and an iPod. Note that it did not allow Windows Mobile
6.5 to display messages by conversation. Iam not aware of how or even if it will
work on any other phones.
GetSearchResults logging fix
============================
A cosmetic fix has been added in GetSearchResults to avoid logging error when GAL
search returns zero matches
SmartFolders - Added wiki folder type
=====================================
While troubleshooting a different issue for a user, I noticed in the log that there
was a wiki folder type on their system. This occurs if the server has been upgraded
all the way from zimbra 5 or earlier. I added entries for the 'wiki' folder type
to the Smart Folders initialization block to accomodate these conditions.
Fake Outbox Now Universal
=========================
The Fake Outbox that was introduced on an experimental basis originally to allow
some newer Nokia phones to sync proved to work well with all newer Nokia phones
without breaking the sync for older Nokia phones.
With the update to android 2.3.3 it seems that android now also needs to see an
Outbox before it will sync correctly to a server.
For this reason, the check of a Nokia UserAgent string has been removed, and now
all devices will get a Fake Outbox folder listed in their Sync setup.
The internal folder name has been changed from NokiaFakeOut to FakeOutbox to
make it clear that this is not longer a Nokia specific feature.
Also, a "path" element was added for the FakeOutbox to fix a debug log error
Save SOAP Errors
================
Code was added to save any SOAP errors extracted from zimbra server responses to
$this->_soapError for decision making in the calling routine. This is used in
conjunction with the Logon retry logic detailed above.
Removed 'Too big for inline?' debug message
===========================================
The 'Too big for inline?' debug message was added to the code during development
to check if the size of attachments was larger than the config.php setting for
MAX_EMBEDDED_SIZE. This debug message has been removed.
Reworked Sendmail address checking
==================================
The checking of the validity of the sender's email address has been reworked.
If the setting ZIMBRA_ENFORCE_VALID_EMAIL is defined and set to true, the code
will now check if the sender's email address is either the user's primary or
a validly configured alias for that same account. It does not look for any other
External Email Accounts that may have been set up by the user in their Web Client.
If the email address is not valid it will be replaced by the user's primary email
address, and a message will be logged to the debug log file.
HTML text handling revised
==========================
The handling for all non-plain text content has been revised in GetMessage. This
should improve the rendering of the emails on the devices
Reverted text/html encoding to base64
=====================================
A previous release had included a change to use quoted-printable encoding for
text/html content of generated emails. This has been reversed, and base64 will
instead be used, as the majority of incoming messages use base64
Added check for 'profile' tags in XML files
===========================================
A check for 'profile' tags in XML files has been added to fix a debug log error
Added quotes around 'rtf'
=========================
Quotes were added around rtf for class_exists checks to fix debug log error
ExtractSessionId and Empty Response
===================================
A check has been added for empty $response content in ExtractSessionID to fix a
debug log error
Changes specifically for as12.1 branch
======================================
Date range handling has been added to the Out-Of-Office logic in getSettings and
setSettings, and a small fix was added for $oofstate logic. as12.1 only.
Some function arguments needed to be changed to work with as12.1 branch - Fetch,
GetMessage & SendMail have all been updated
The handling of Follow-Up flags was changed to work with as12.1 branch
The classes ExportChangesZimbra and ImportHierarchyChangesZimbra were removed as
they are not needed for as12.1
IF YOU ARE UPGRADING FROM RELEASE 48 OR EARLIER - IT IS CRITICAL THAT YOU READ THE
RELEASE NOTES FOR RELEASES 49 & 50. SOME MAJOR CHANGES WERE IMPLEMENTED IN THOSE
RELEASES (ROLLED UP INTO RELEASE 51 ALSO) THAT WILL AFFECT YOUR SETUP
==================================================================================
Changes Made To Revision 50:
- Removed use of Mail_MimeDecode for synching emails out to the device, and
for getting details of the original email for Smart Forward/Reply as it
was found to be consuming huge chunks of memory whenever an email had large
attachments. This could cause the sync to crash. It would continue to
crash every time the phone tried to sync the mail folder until such time
as the message was moved on the server, or was outside of the window for
days to sync. Replaced functionality with native zimbra soap calls.
Mail_MimeDecode is still needed for SendMail from the phone to handle the
raw rfc822 data sent to zimbra - hopefully people will not be sending
huge attachments from their phones but, even if the send does fail, it
will not affect the sync of emails out to the phone.
- Fixed filtering Calendar/Contacts/Task folders when virtual is set to false
for a folder type and SmartFolders is enabled
- Removed function FromHeaderFix - as it is not used
- Removed function GetAttachmentBody - as it is not used
- Removed function GetBody - as it is not used
- Changed text/html encoding from base64 to quoted-printable for mime body
as it generally produced smaller output
- Added check for 'profile' in XML file to fix debug log error
- Added quotes around rtf for class_exists checks to fix debug log error
- Added check for empty $response content in ExtractSessionID to fix debug
log error
- Added Date range handling to Out-Of-Office logic in getSettings and
setSettings, and fixed $oofstate logic.
- Changed function arguments to work with as12.1 branch - Fetch, GetMessage,
SendMail
- Changed handling of flags to work with as12.1 branch
- Removed class class ExportChangesZimbra as not needed for as12.1
- Removed class class ImportHierarchyChangesZimbra as not needed for as12.1
Changes Made To Revision 49:
- HTML email when used with a patched install of the as12 branch of z-push
from the z-push SVN repository. I will document the installation process
separately in the near future. It has been tested with Android 2.3 and
Windows Mobile 6.5 to date. I have no access to Apple devices so cannot
do any testing on those - so they may still need some work.
The remainder fo the changes below apply to installations under either an
official z-push 1.5.x release, or the patched as12 release.
- Added new options to the config file to allow disabling of any folder
type system-wide. See details in notes at top of zimbra.php file
- Added new SmartFolders feature - to allow users to control the list of
folders they wish to sync to their devices without using XML files
NOTE: If you wish to enable SmartFolders - you MUST stop sync and clear
all old state data from the state folder. Even if no SmartFolder names
are used, the sequence and naming of the folders will change
NOTE: If SmartFolders are enabled, XML files will not be processed
To enable the feature set ZIMBRA_SMART_FOLDERS to true in config.php
Once enabled, the final character in a folders name can take on special
meanings as follows :-
"-" Do not include this folder or any subfolders thereof
"." Include this folder - But do not include any subfolders thereof
- Changed Attachments handling to allow attachments in multi-part messages
to be returned. Also allows attaching of an email as a .eml file
- Added ability to filter out Completed Tasks during sync - Note: As zimbra
does not store a Completed Date, the last modified timestamp is used
- Added Reminders for Tasks (for zimbra 7.0+) - need a zcs 6.0.nn user to test
if it will work on zcs 6.0.nn also
- Enhancement in SendMail to include original message text for reply/forward
This is done in preference to attaching the original as Android devices are
unable to open .eml attachments natively at this time
- Added check in SendMail for z-push version to ensure we properly format the
path to the "state" directory - change is to match the config.php change to
the definition of STATE_DIR introduced by z-push 1.5
- Fix in SendMail to honour ZIMBRA_ENFORCE_VALID_EMAIL when set to true. If
the from email address used is not a valid one for the account it is
replaced by the default sender email address.
- Fix in SendMail to add sender name to the address in all cases. This is a
generalized fix based on the original premise of the ZIMBRA_NOKIA_MFE_FIX
as it seems that iPhone and Android also send emails without a sender name
- Fixed GetSearchResults to match response 'keys' to latest zpushdefs.php
- Fixed possible memory leaks and reduced memory usage by unset()ing rtf and
mime decoder class instances, and various arrays
- Removed the input parameter from the params array send to Mail_mimeDecode
as it is already passed to the class on creation, and created and unset a
params object to ensure that memory is cleaned up
- Fix for special characters (",&,<,>) in user passwords
- Added fix for Euro symbol & other non-standard cp1252 characters
- Changes made to MoveMessage to prevent items being moved from one account to
another, and from one folder type to another. These restrictions are to
prevent items becoming inaccessible on the device. See comments in function
for more information.
- Removed username from Sync Request log entry for z-push 1.5 and later as
the z-push engine is already logging it
- Fix for Categories field from HTC Desire (Android 2.2) - Phone sends an
empty string Categories field instead of an array. The blank field caused an
error in CreateTagRequest. Fix to check that (is_array(input->categories))
- Fix added for iPhone/iPad/etc Meeting Request/Response handling as it appears
that the Apple client sends off meeting requests/responses directly from the
handset instead of letting the server handle them
- Fix in PROVISIONING getPolicyKey to avoid writing "validate" records in
database for android devices
- Fix in PROVISIONING generatePolicyKey to limit upper bound on random number
generator to 2147483647 (as some phones cannot handle larger numbers)
- Added function ReportMemoryUsage (called on Logoff) to track memory use
- Added new functions that are needed for ActiveSync protocol version 12
- ImportMessageFlag - use of this required a change to request.php
- GetSettings - used for Out-Of-Office
- SetSettings - used for Out-Of-Office, and storing Handset data
- Updated GetSearchResults to add mailbox search capability
PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 50
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES
Minimise use of Mail_MimeDecode - to avoid running out of memory
================================================================
Shortly after publishing Release 49 an issue was discovered when handling an
email with a 13MB attachment. The loading of the email, and processign it with
Mail_MimeDecode resulted in the session running out of memory. The session
crashed, and reported a memory allocation error in the Apache log file.
Prior to release 49, this function was only used once in the application - at
the point where the user is sending an email from the phone. It is used to
process the raw uploaded file to prepare it for zimbra to send.
Release 49 introduced it's use to other places, synching emails out to the
device, and for getting details of the original email for Smart Forward/Reply.
It was in these places where it was found to be consuming huge chunks of
memory whenever an email had large attachments. This could cause the sync to
crash. It would continue to crash every time the phone tried to sync the mail
folder until such time as the message was moved on the server, or was outside
of the window for days to sync.
Release 50 has replaced the functionality for which this function was used
with native zimbra soap calls.
Mail_MimeDecode is still needed for SendMail from the phone to handle the raw
rfc822 data sent to zimbra as it was in releases prior to 48 - hopefully
people will not be sending huge attachments from their phones but, even if
the send does fail, it will not affect the sync of emails out to the phone.
Fixed filtering Calendar/Contacts/Task virtual folders
======================================================
When ZIMBRA_VIRTUAL_<Folder type> is set to false for a folder type with
SmartFolders enabled, the folder list was not being filtered correctly. This
has been fixed, and should now be working as expected.
System-wide Disable Switches for Folder Types
=============================================
Several users have asked over time if it is possible to turn off Email sync or
Calendar sync for an entire installation. Up to now it has only been possible
through a hack to force all users to read a particular XML file.
This release makes it possible to turn off individual folder types through the
config file. Enabling any of these switches will turn off that folder type (for
example email) for every user of the system. It will prevent all users from
syncing their email to their mobile devices.
The default setting remains that these switches are either not present - or are
set to false - in which case all folder types will be available for synching.
=== To Disable Sync of any Folder Type System-wide ===
Add the appropriate definition from this block to config.php, and set it true
NOTE: Devices will need a full-resync to remove the unwanted content
define('ZIMBRA_DISABLE_MESSAGES',true);
define('ZIMBRA_DISABLE_CONTACTS',true);
define('ZIMBRA_DISABLE_APPOINTMENTS',true);
define('ZIMBRA_DISABLE_TASKS',true);
SmartFolders
============
WARNING TO SYSTEM ADMINISTRATORS. IF YOU WISH TO ENABLE SMART FOLDERS YOU MUST STOP
SYNC FOR ALL DEVICES, CLEAR OUT THE STATE DIRECTORY, AND THEN RESYNC ALL DEVICES
THIS IS DUE TO AN OVERHAUL IN THE NAMING AND SEQUENCING OF FOLDER IDs
For many releases now, there has been the ability to manipulate the number and types
of folders that could be synced for an individual user through the use of user.XML
configuration files. While this worked well, it meant that the system administrator
had to be involved in every change for every user.
This release introduces a new feature called SmartFolders as an alternative to XML
files that will hopefully make it easy for users who wish to manipulate their syncing
content to be able to do the job themselves.
Note that SmartFolders and XML files are mutually exclusive on the server. Enabling
SmartFolders disables the use of user.XML files for everyone.
To enable the feature set ZIMBRA_SMART_FOLDERS to true in config.php
Once enabled, the final character in a folders name can take on special meanings as
follows :-
"-" Do not include this folder or any subfolders thereof
"." Include this folder - But do not include any subfolders thereof
So for example you might have a top level folder called "Archive-" into which you
move all old folders that you do not want to be able to see from your device.
Or you might have a "ToBeFiled." folder that contains a number of child folders used
for longer term storage of reference emails. Naming it with a period (.) on the end
will allow you to see that folder on your phone - so you can move emails in there to
clear your Inbox - and then when you get to your desktop you can file all the emails
into the appropriate child folders.
The SmartFolders feature is particularly useful on Android devices - which list out
every folder from your server across the top of the email client.
=== To enable the Smart Folders feature ===
NOTE: Enabling SmartFolders will DISABLE user.XML processing
It will also rename/resequence folder ID's in the state file for
the device - so state directory must be cleared prior to it's use
define('ZIMBRA_SMART_FOLDERS',true);
User control of Folder types
----------------------------
NOTE: z-push does not do hierarchy resync properly at this time - so if a user wishes
to make changes to their sync rules using the following directives, they should first
remove the sync account from their phone - then make the changes - then re-add the
account to their phone.
When SmartFolders are configured, in instances where individual users wish to disable
particular folder types from syncing to their devices, a special top-level folder
structure can be used to configure those options. This provides the major filtering
capability that would have been available through user.XML files.
If a folder named '*SyncConfig*' is found, the system will use any child folders in
that folder to configure options.
The names of the sub-folders will be interpreted as configuration directives.
These should have the format of
<folder type>&<setting=value>[&<setting=value>& ...]
<folder type> can be any one of message, contact, appointment, task and
<setting=value> can be any one of
active=true/false - active=false will disable sync of that folder type
virtual=true/false - virtual=false will turn of sync of additional folders (not
applicable for message type)
primary=FolderName - setting primary will override the default primary folder for
that content type (Inbox, Contacts, Calendar, Tasks)
So, for example, you could have
*SyncConfig*
message&active=false to disable email sync
task&active=true&virtual=false to limit the task sync to the primary folder
appointment&active=true&virtual=false&primary=WorkCalendar to limit
appointment sync to the WorkCalendar folder
The *SuncConfig* folder, and it's contents will never be synced to the phone so
long as ZIMBRA_SMART_FOLDERS is set to true.
Improvements in Attachment Handling
===================================
Attachments handling has been completely re-written to allow attachments in multi-part
messages to be returned. This restructuring also has an added benefit in that it
allows attaching of an email as a .eml file to another email.
Note: Android is currently unable to open attached email messages - but other clients
should be able to do so.
Improvements in Task Sync
=========================
Added ability to filter out Completed Tasks during sync - Note: As zimbra does not
store a Completed Date, the last modified timestamp is used
Added Reminders for Tasks (for zimbra 7.0+) - I no longer have a zcs 6.0.nn install
so if someone running version 6 can uncomment the 7.0 restriction and test it would
be very much appreciated.
Improvements in SendMail
========================
The entire SendMail function has been re-written to decompose the mime message
received from the phone, and to recompose it prior to sending. This allows for
easy fixing of the from address, as well as implementing SmartForward and
SmartReply (where the phone does not send the original content with the reply/
forward text)
The major enhancement in SendMail is to include original message text for reply/
forwarded emails. This is done in preference to attaching the original as Android
devices are unable to open .eml attachments natively at this time. In a future
release I may look to make this optional - or to follow the configuration in the
users Web client preferences
Added check in SendMail for z-push version to ensure we properly format the path
to the "state" directory - change is to match the config.php change to the
definition of STATE_DIR introduced by z-push 1.5 - The emails to be send are first
uploaded to the state directory, and then send from there by zimbra.
Fix in SendMail to honour ZIMBRA_ENFORCE_VALID_EMAIL when set to true. If the
from email address used is not a valid one for the account it is replaced by the
default sender email address.
Fix in SendMail to add sender name to the address in all cases. This is a
generalized fix based on the original premise of the ZIMBRA_NOKIA_MFE_FIX as it
seems that iPhone and Android also send emails without a sender name
Search should now return proper results
=======================================
Fixed GetSearchResults to match response 'keys' to latest zpushdefs.php
Company searches should now return lists of users as expected.
Improvements in Memory Management
=================================
Fixed possible memory leaks and reduced memory usage by unset()ing rtf and mime
decoder class instances, and various arrays
Note: even with these improvements, the memory footprint of an individual session
has increased slightly as a result of all the new functionality
Removed the input parameter from the params array sent to Mail_mimeDecode as it
is already passed to the class on creation, and created and unset a params object
to ensure that memory is cleaned up
Fix for Special Characters in Passwords
=======================================
Fix for special characters (",&,<,>) in user passwords. These characters should
no longer prevent users from synching.
Fix for Euro Symbol and Other cp1252 Characters
===============================================
Added fix for Euro symbol & other non-standard cp1252 characters that are not
represented in UTF-8
Restricted MoveMessage to single accounts/folder types
======================================================
Changes made to MoveMessage to prevent items being moved from one account to
another, and from one folder type to another. These restrictions are to
prevent items becoming inaccessible on the device.
It seems that when you move a message from a folder shared with you into one of
your own folders, or vide-versa, zimbra assigns it a completely new message id.
However, there is no way to retrive what that new message id is through the SOAP
call. Therefore we ended up with a message now in the receiving folder that the
phone thinks still has it's original id from the originating folder. However, on
the server the id has changed. The item id on the phone therefore does not exist
on the server - and as a result cannot be moved/deleted/manipulated on the phone.
It will just sit there until the sync window (3 days for example) causes it to
drop out of the phone.
To avoid this situation - the phone is now prevented from moving items from one
users folder to another user's folder. Any attempt to move an item in this way
will return an error to the phone, and will log a message to the debug.txt file.
Tidied up SyncRequest log entries
=================================
Removed username from Sync Request log entry for z-push 1.5 and later as the
z-push engine is already logging it
HTC Android - Categories Fix
============================
Implemented a fix for Categories field from HTC Desire (Android 2.2)
In the case where a modified item on the phone, has no categories associated
with it, the phone sends an empty string in the Categories field. This is unexpected
in two ways. Firstly, most phones simply omit the field as it is optional. Secondly,
it sends a string variable instead of an array of string variables. The blank
field caused an error in CreateTagRequest.
The fix is to check that (is_array(input->categories)) prior to calling the
CategoriesToTags function.
Apple iPhone/iPad Meeting Request/Response Fix
==============================================
Note: As I do not own or have access to any Apple devices, I have been limited to
third party testing on this. It may still need more work.
A fix has been added for iPhone/iPad/etc Meeting Request/Response handling as it
appears that the Apple client sends off meeting requests/responses directly from the
handset instead of letting the server handle them. This was resulting in duplicated
emails going to and from the attendees and meeting organizer.
Hopefully the behaviour will be much better with these changes.
Improvements in PROVISIONING handling
=====================================
Fix in PROVISIONING getPolicyKey to avoid writing "validate" records in
database for android devices
Fix in PROVISIONING generatePolicyKey to limit upper bound on random number
generator to 2147483647 (as some phones cannot handle larger numbers)
Added Memory Use Reporting Function
===================================
Added function ReportMemoryUsage (called on Logoff) to track memory use
Added new Functions needed for ActiveSync 12 protocol
=====================================================
Added new functions that are needed for ActiveSync protocol version 12
- ImportMessageFlag - use of this required a change to request.php
- GetSettings - used for Out-Of-Office
- SetSettings - used for Out-Of-Office, and storing Handset data
- Updated GetSearchResults to add mailbox search capability
============== Z-Push Zimbra Backend - Release Notes - Revision 50 ==============
======================== CHANGES MADE IN EARLIER RELEASES ========================
Changes Made To Revision 48:
- Critical fix for v5 compatability fix. LookupV5Timezone missing the
$this-> pointer, so calls to the function caused crash in backend.
- Fix typo in name of v5timezone.xml file
- Added more error handling/logging around zimbra 5 compatability code, and
updated the v5 compatability notes
- Change to ExtractSessionID to return session for zimbra 5
- Change AlterPing to always return false for zimbra 5 as i4ms flag used by
AlterPingChanges does not exist on zimbra 5. Old folder ping method must
be used
- Fix in GetZimbraFolders to check for a folder identifier (l) before adding
the folder to the folder list. This is to avoid including any folders from
the notify block in the soap response
- Moved setting of _wasteID out of GetFolder and into Setup as GetFolder is
only called on a Hierarchy Sync - not on every sync - so it ended up being
set to false more often than not
- Fix in GetMessage to not create another meeting request out of a meeting
acceptance email
- Fix in CreateMeetingRequest to add invitees to the meeting if they are
provided, and to send out meeting invitations to the email addresses
specified (Android sends them - Nokia/WM don't - Not sure about iPhone)
- In function GetBody added html_entity_decode and preg_replace for duplicate
blank lines to make plain text output more readable when there is no
text/plain body part in the message
- Moved PROVISIONING: getDeviceRWStatus log message so it only logs for
unrecognised devices or those that have had a device wipe requested
- Added option to define ZIMBRA_DEBUG as 'user1' or 'user1,user2,userN' to
enable zimbra soap debug messages for just the listed accounts. Defining
it as true/false will continue to work as before
Changes Made To Revision 47:
- Added NokiaFakeOut to allow Nokia E72 and some other models to Send emails
- Added compatability fix for zimbra 5 timezone handling (see notes below)
- Changed ConvertItemIdToInvID to check for shared item IDs (i.e. look for a
":" in the passed $id) in case user modifies a shared appt on phone
- removed l= specified from ModifyAppointmentRequest to avoid moving a shared
appt to user's own Calendar when editing it on the phone
- Fix to avoid replacing the organizer name/email in ModifyAppointmentRequest
- Check for Organizer Name exists before outputting (to avoid error in log)
- Fix in GetBodyRecursive to use mb_detect_encoding() to identify the original
encoding of an email - prior to converting it to UTF-8 for the phone. To use
mb_detect_encoding, the PHP extension package php-mbstring must be installed
Changes Made To Revision 46:
- Added new functions TagsToCategories & CategoriesToTags to support syncing
Categories<->Tags on all content types
- Extended Categories<->Tags syncing to Appointments/Tasks (Note: z-push doesn't
support categories on Message sync in z-push 1.4)
- Fixed loop counter in Setup function to prevent recursion through all newly
added folders when checking for children of linked folders
- In StatMessage function for 'message' replaced call to GetMsgRequest with call
to GetMsgMetadataRequest to reduce data transfer and logging
- Improved sync of Birthday/Anniversary - Dates on zimbra must be in YYYY-MM-DD
format (or if --MM-DD format this/next year will be added depending on date)
- Fixed Children sync on Contacts by treating the value as an array instead of
a string. Uses implode/explode (Note: does not sync to Nokia Children field)
- Hard linked webpage sync on Contacts to workURL on zimbra as only one of the 3
workURL, homeURL and otherURL can be synced out to the phone, and once sync'ed
if a change is made to the contact on the phone, we would not know which of the
three URL fields to sync webpage to on an incoming contact
- Fix to avoid overflowing 32-bit integer for calExpandInstStart and
calExpandInstEnd when syncing calendar/tasks. Use strval() and append "000"
instead of multiplying by 1000
- Replaced divide by 1000 with substr() to avoid conversion issues in
Date4ActiveSync and GetMessage (appointment request email)
- Added new CONFIG.PHP key ZIMBRA_DEBUG to enable/disable logging of the SOAP
requests & responses, and the Setup Folder Lists. This allows more granular
debugging than using on the value of the WBXML.PHP key WBXML_DEBUG that turns on
Z-Push internal logging. It is now possible to log either on their own, or both
- Fix in GetMessage to avoid e_attribute_?? arrays to match names with email
addresses, as a blank value for a persons name would result in the incorrect
Personal name being matched with any subsequent email addresses
- Removed unused code and debug statements, and tidied up code
- Added DeviceType on new, and updated DeviceType and UserAgent on sync in
getDeviceRWStatus to show most recent client version that sync'ed
- Fixed reference to this_WasteID instead of $this->_wasteID
Changes Made To Revision 45:
- Implemented the provisioning "Remote Device Wipe" functionality. This needs a
MYSQL database to work. To disable it, set PROVISIONING to false in config.php
- Sync Device Categories with Server Tags (requirement for zimbra 7)
- Removed html="1" from GetMsgRequest calls in StatMessage to reduce data returned
- Fixed getBody to compare on passed $html variable instead of $this->_useHTML
- Added comments & Reduced logging in AlterPingChanges
Changes Made To Revision 44:
- Fixed parameters on iconv call added by Release 43 to handle foreign languages
Changes Made To Revision 43:
- Implemented the new AlterPing/AlterPingChanges method from 1.4 to reduce the
number of hits on the backend for monitored folders. Operates on the basis of
monitoring the i4ms flag on folders for changes
- Added handling of RTF field in appointment/task/contact where no Body/Notes
field received. Requires z_RTF.php include file from as12 branch of z-push SVN
to decode the Compressed RTF stream. Issue reported by Danish language user
- Removed multiple classes/functions that were duplicates of those in diffbackend
to simplify the updating of the code to match future z-push releases. We should
only declare "standard" classes/functions where we have a need to override.
- Changed debug logging of Soap Request/Response to follow WBXML.PHP debug flag
so that basic application flow debugging will not flood logs
- Changed date selection criteria for Appointments/Tasks to use the attributes
calExpandInstStart and calExpandInstEnd where Start is determined by cutoffdate
or defaulted to 366 days in the past and End is arbitrarily set to 366 days in
the future
- Fixed some issues with recurring appointments on last day or named day of month
Outgoing appointments would set WeekOfMonth to -1 when ActiveSync wants a value 5
Incoming appointments would set bysetpos to 5 where zimbra wants a value -1
- Changed default for all folder types to recursive=true - XML will still override
Changes Made To Revision 42:
- Fixed typo in $this->_virtualAddressbook missing an "s" on end
- Added isset checking to clean up "undefined index" entries in debug log
Changes Made To Revision 41:
- Fixed Bug With SendAsEmail and SendAsName
Changes Made To Revision 40:
- Added 'function_exists' Check for Timezone Functions
- Fixed Some Profile Parsing Issues
- Fixed Bug Where '&' in Subject or Body of Appointment Would Cause it Not to Sync
Changes Made To Revision 39:
- Removed PHP-SOAP From Requirements
- Added Error Checking For PHP-CURL
- Changed Tag For Search Folders In User File
- Meeting Organizer Name & Email Uses SendAsName and SendAsEmail From User File. If Not Specified, Get From Zimbra
- Added Config File Option (ZIMBRA_ENFORCE_VALID_EMAIL) To Prevent User (True) From Using Email Address (SendAsEmail) Not Owned By Them
Changes Made To Revision 38:
- Search Folder Support Added For Mail
- Fixed Bug In Profile Loading
- Fixed Bug That Was Not Handling The Absense Of The "Optional" AllDayEvent Flag From The Phone (Palm Pre Issue)
- Fixed Bug In AllDay Appointments Timezone
- Fixed Bug Where Multi-Day AllDay Server Appointment Appearing As Single Day On Phone
- Fixed Bug In Generating Timezone Blob From Zimbra Appointment That Caused Recurring Appointments To Move
- Fixed Meeting Request Emails So They Request Response (Accept/Decline/Tentative)
- Implemented MeetingResponse To Handle Meeting Requests (Requires Z-Push 1.4 Or Later)
- Refactored To Use $tzObject To Identify Every Instance Of A Timezone Identifier For Converting To/From ActiveSync
(Excessive Use Of $tz Was Getting Confusing)
- Removed Some Nokia Specific Code That Is No Longer Needed Now Timezones Are Working Better
Changes Made To Revision 37:
- NOTE TAG CHANGES TO USER PREFERENCE FILES
- Added Ability For Multiple Profiles In User Preference File (See Below For Details)
- Fixed Timezone Handling For Calendar Appointments
- Added Ability To Specify Roaming Timezone (See Below For Details)
- Tasks Implemented - Note: Zimbra supports a limited set of the ActiveSync Task functionality
(eg. Date Completed/Reminders/Repetition/Categories are not supported)
Credits
=======
This php script makes use of mime.php and mimePart.php from the PEAR project
Download Latest Version
zimbra73.tgz (224.7 kB)
