============== Z-Push Zimbra Backend - Release Notes - Release 51 ==============
THESE RELEASE NOTES FOR RELEASE 51 CHANGES
PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 51
NOTE THAT THE RELEASE NOTES FOR 49 & 50 ARE ALSO CONTAINED BELOW AND SHOULD
ALSO BE READ FULLY WHEN UPGRADING FROM A RELEASE 48 OR EARLIER.
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES
AND ARE UPGRADING AN EXISTING INSTALLATION FROM RELEASE 48 OR EARLIER.
Changes Made To Revision 51:
- Added attendee status/type information to appointments
- Changed GlobalObjID to be correct non-outlook encoded value so that Apple
and possibly other devices will update appointments correctly
- Changed handling of invitations for client generated meetings - zimbra will
always send out invitations now (even for Apple/Android) - but if zimbra is
the default email client on the device, we will drop the device originated
invitation once it is received by SendMail. If another email account is your
default account then the device will send a "duplicate" meeting invitation
directly through that email account. Replies to the device originated
invitation may not correctly update the attendee status of the appointment
on the original device
- Fixed as12+ body handling for Appointment/Contact/Task
- Changed handling of birthday/anniversary field to try to get a solution
that will work for most scenarios
- android 2.3.3 seems to have a serious problem with the birthday field that
causes it to spin constantly trying to resync the calendar - limiting
birthday to known UserAgents for now (Apple and MailforExchange)
- Experimental fix in Logon to delay and retry logon attempts when the
zimbra server is not available (eg. zimbra FOSS shut down for backup)
See ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR in the notes below.
- Fix to add missing $this-> prefix to ScrubHtmlText in function
GetMpBodyRecursive
- Added fix to AlterPingChanges to detect changes in Virtual Folders
- Added output the Conversation ID field to allow view by conversation
on some Android devices (WM appears to need more to work)
- Cosmetic fix in GetSearchResults to avoid logging error when GAL search
returns zero matches
- Added 'wiki' folder type for SmartFolder setup to accomodate servers that
are running, or once ran, version 5.0.x of zimbra
- Removed Nokia only filter on adding FakeOutbox folder, as Android 2.3.3
seems to need it too, and it doesn't appear to break other phone syncs
and renamed the devid to FakeOutbox from NokiaFakeOut
- Added path to fake outbox structure to fix debug log error
- Save SOAP error extracted from response to $this->_soapError for
decision making in calling routine
- Removed 'Too big for inline?' debug message
- Reworked Sendmail address checking
- Revised non-plain text email handling in GetMessage
- Reverted text/html encoding to base64 from quoted-printable for mime body
as the majority of incoming messages use base64
- Added check for 'profile' in XML file to fix debug log error
- Added quotes around rtf for class_exists checks to fix debug log error
- Added check for empty $response content in ExtractSessionID to fix debug
log error
- Added Date range handling to Out-Of-Office logic in getSettings and
setSettings, and fixed $oofstate logic.
- Changed function arguments to work with as12.1 branch - Fetch, GetMessage,
SendMail
- Changed handling of flags to work with as12.1 branch
- Removed class class ExportChangesZimbra as not needed for as12.1
- Removed class class ImportHierarchyChangesZimbra as not needed for as12.1
Attendee Status & Type
======================
The status (Accepted/Tentative/Declined/Unknown) and type (Required/Optional/Resource)
information for meeting attendees are now pushed out to the device.
Changed Global Object ID handling
=================================
In some cases, zimbra meeting updates were not properly updating the client device.
Changed the handling of the GlobalObjId field to encode it as a "non-Outlook" ID to
improve the update reliability
Meeting Requests and Duplicate Invites (Apple and Android??)
============================================================
Having struggled to understand the behaviour of the Apple iP*** devices and their
handling of meeting requests, I was finally able to get access to an iPod (4.3) to
test with.
What I discovered is that whenever you set up a meeting on an Apple device, the
device will create the meeting on the server, and once it gets confirmation that the
meeting was created successfully, it will then send out emails directly from the
device to the invitees in a 3-part MIME message inviting them to the meeting. The
third part of the MIME message is of type text/calendar and contains an ICS meeting
request. Zimbra by default also sends out meeting requests - so the recipient will
end up receiving 2 separate invites to the same meeting.
I also found that if I accepted the meeting request sent from the device directly,
it did not update properly update the meeting attendee status for the meeting.
For these reason, I have decided to adopt the following processing rules.
1. zimbra will be allowed to invite participants to all meetings
2. if the device-originated invitations are sent through zimbra - I will drop them
without actually sending the email to the recipient.
This apporach gives the best outcome in terms of allowing for correct handling of
replies to the meeting request, and status updates, etc.
There is still no way to intercept the duplicate meeting invitation from the device
however, if it is configured with multiple email accounts, and it routes the
direct invitation out through one of the alternatie accounts. I believe you can
select the "Default Account" in the Email settings on the Apple devices to force
email to go through the ActiveSync account as a preference - which would allow all
duplicate invites to be caught and dropped.
Fix AS12+ Body Handling
=======================
Just as AS12+ brought with it the ability to specify the preferred content of emails
it also brought the same ability to Appointment/Contact/Task body fields. This release
fixes the handling of the body field when an AS12+ server is being used.
Changed handling of Birthday/Anniversary
========================================
In troubleshooting a birthday field issue raised by an iPhone user, I found the
original scheme employed in the code to try to ensure the date conversion would always
work was causing issues with the iPhone. Whatever date the user set on zimbra would
get incremented by one day on the iPhone. After some experimentation I think I have
found an alternative thet will work more reliably with most if not all phones. This
has been implemented. (See also Birthday Field Breaks Android 2.3.3 Sync below)
Birthday Field Breaks Android 2.3.3 Sync
========================================
Having done extensive troubleshooting over a number of months on an issue where my
own phone, having been updated to android 2.3.3, began to constantly sync the
calendar regardless of the ActiveSync "ping" settings, I finally stumbled on the
solution to the problem while working on the iPhone Brithday issue above.
It seems that Android 2.3.3 does not like the birthday field as presented by z-push
/zimbra. I have no way to know if a real Microsoft Exchange Server birthday field will
behave the same way - but I assume it will. If the birthday field is present on the
server, the Calendar on the phone will constantly sync - connecting every minute -
which will run up data charges and drain the battery rapidly on the device. I assume
the calendar is trying to look for, or create a "birthday" appointment with which to
sync - but nothing happens other than the constant spinning.
For this reason, I have limited the devices that the birthday field will sync to to
those devices I know do not have such problems. Nokia (MailforExchange) and Apple
are the only unique UserAgent strings I have been able to allow to work for now.
For any other device, the contact record will still sync, just the birthday field
will not be sent to the device.
Experimental fix in Logon to delay and retry logon
==================================================
Like many Zimbra FOSS users, we use a backup script from the Administrators forum
that requires zimbra to be shutdown briefly to do a cold-sync backup. For some
mobile devices, if they try to initiate a connection to the server during that
period, they will spin quickly retrying and will then give-up. Other devices try
again after a period of time has elapsed.
This experimental fix has been put in place to allow a few minutes for the server
to become available again so that the device does not get into this "I've given up"
state. It has been working well for me.
To enable the functionality ass a new definition in the config.php file
define('ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR',5);
and set the value appropriately for your system.
I am not aware of the tolerance of phones towards communication delays - hence the
experimental label.
Fix for ScrubHtmlText
=====================
A fix has been implemented to add missing $this-> prefix to ScrubHtmlText in function
GetMpBodyRecursive. This bug was causing the script to terminate in the event that
the phone requested text output, but the email was in HTML only.
Fix for "Ping"ing Virtual Folders
=================================
A fix was added to AlterPingChanges to detect changes in Virtual Folders. These
folders are treated as extensions of the main Calendar/Tasks/Contacts folders,
and their content is synched over to the equivalent folders on the phone. However,
an issue was identified whereby a change to an item in the virtual folder would not
trigger a resync of the folder. This has been fixed.
A word of warning for anyone who shares folders among very large groups of users is
that all users would get the notification to sync within the same 5 second time
window - and this could add extra load spikes to your Apache and/or Zimbra servers.
Conversation View (partial support)
===================================
While zimbra does not support the full conversation view history that Exchange does,
adding the Conversation ID at least allows some phones to group emails together
correctly by Conversation. This has been tested under the as12.1 configuration
using an android 2.3.3 phone and an iPod. Note that it did not allow Windows Mobile
6.5 to display messages by conversation. Iam not aware of how or even if it will
work on any other phones.
GetSearchResults logging fix
============================
A cosmetic fix has been added in GetSearchResults to avoid logging error when GAL
search returns zero matches
SmartFolders - Added wiki folder type
=====================================
While troubleshooting a different issue for a user, I noticed in the log that there
was a wiki folder type on their system. This occurs if the server has been upgraded
all the way from zimbra 5 or earlier. I added entries for the 'wiki' folder type
to the Smart Folders initialization block to accomodate these conditions.
Fake Outbox Now Universal
=========================
The Fake Outbox that was introduced on an experimental basis originally to allow
some newer Nokia phones to sync proved to work well with all newer Nokia phones
without breaking the sync for older Nokia phones.
With the update to android 2.3.3 it seems that android now also needs to see an
Outbox before it will sync correctly to a server.
For this reason, the check of a Nokia UserAgent string has been removed, and now
all devices will get a Fake Outbox folder listed in their Sync setup.
The internal folder name has been changed from NokiaFakeOut to FakeOutbox to
make it clear that this is not longer a Nokia specific feature.
Also, a "path" element was added for the FakeOutbox to fix a debug log error
Save SOAP Errors
================
Code was added to save any SOAP errors extracted from zimbra server responses to
$this->_soapError for decision making in the calling routine. This is used in
conjunction with the Logon retry logic detailed above.
Removed 'Too big for inline?' debug message
===========================================
The 'Too big for inline?' debug message was added to the code during development
to check if the size of attachments was larger than the config.php setting for
MAX_EMBEDDED_SIZE. This debug message has been removed.
Reworked Sendmail address checking
==================================
The checking of the validity of the sender's email address has been reworked.
If the setting ZIMBRA_ENFORCE_VALID_EMAIL is defined and set to true, the code
will now check if the sender's email address is either the user's primary or
a validly configured alias for that same account. It does not look for any other
External Email Accounts that may have been set up by the user in their Web Client.
If the email address is not valid it will be replaced by the user's primary email
address, and a message will be logged to the debug log file.
HTML text handling revised
==========================
The handling for all non-plain text content has been revised in GetMessage. This
should improve the rendering of the emails on the devices
Reverted text/html encoding to base64
=====================================
A previous release had included a change to use quoted-printable encoding for
text/html content of generated emails. This has been reversed, and base64 will
instead be used, as the majority of incoming messages use base64
Added check for 'profile' tags in XML files
===========================================
A check for 'profile' tags in XML files has been added to fix a debug log error
Added quotes around 'rtf'
=========================
Quotes were added around rtf for class_exists checks to fix debug log error
ExtractSessionId and Empty Response
===================================
A check has been added for empty $response content in ExtractSessionID to fix a
debug log error
Changes specifically for as12.1 branch
======================================
Date range handling has been added to the Out-Of-Office logic in getSettings and
setSettings, and a small fix was added for $oofstate logic. as12.1 only.
Some function arguments needed to be changed to work with as12.1 branch - Fetch,
GetMessage & SendMail have all been updated
The handling of Follow-Up flags was changed to work with as12.1 branch
The classes ExportChangesZimbra and ImportHierarchyChangesZimbra were removed as
they are not needed for as12.1
IF YOU ARE UPGRADING FROM RELEASE 48 OR EARLIER - IT IS CRITICAL THAT YOU READ THE
RELEASE NOTES FOR RELEASES 49 & 50. SOME MAJOR CHANGES WERE IMPLEMENTED IN THOSE
RELEASES (ROLLED UP INTO RELEASE 51 ALSO) THAT WILL AFFECT YOUR SETUP
==================================================================================
Changes Made To Revision 50:
- Removed use of Mail_MimeDecode for synching emails out to the device, and
for getting details of the original email for Smart Forward/Reply as it
was found to be consuming huge chunks of memory whenever an email had large
attachments. This could cause the sync to crash. It would continue to
crash every time the phone tried to sync the mail folder until such time
as the message was moved on the server, or was outside of the window for
days to sync. Replaced functionality with native zimbra soap calls.
Mail_MimeDecode is still needed for SendMail from the phone to handle the
raw rfc822 data sent to zimbra - hopefully people will not be sending
huge attachments from their phones but, even if the send does fail, it
will not affect the sync of emails out to the phone.
- Fixed filtering Calendar/Contacts/Task folders when virtual is set to false
for a folder type and SmartFolders is enabled
- Removed function FromHeaderFix - as it is not used
- Removed function GetAttachmentBody - as it is not used
- Removed function GetBody - as it is not used
- Changed text/html encoding from base64 to quoted-printable for mime body
as it generally produced smaller output
- Added check for 'profile' in XML file to fix debug log error
- Added quotes around rtf for class_exists checks to fix debug log error
- Added check for empty $response content in ExtractSessionID to fix debug
log error
- Added Date range handling to Out-Of-Office logic in getSettings and
setSettings, and fixed $oofstate logic.
- Changed function arguments to work with as12.1 branch - Fetch, GetMessage,
SendMail
- Changed handling of flags to work with as12.1 branch
- Removed class class ExportChangesZimbra as not needed for as12.1
- Removed class class ImportHierarchyChangesZimbra as not needed for as12.1
Changes Made To Revision 49:
- HTML email when used with a patched install of the as12 branch of z-push
from the z-push SVN repository. I will document the installation process
separately in the near future. It has been tested with Android 2.3 and
Windows Mobile 6.5 to date. I have no access to Apple devices so cannot
do any testing on those - so they may still need some work.
The remainder fo the changes below apply to installations under either an
official z-push 1.5.x release, or the patched as12 release.
- Added new options to the config file to allow disabling of any folder
type system-wide. See details in notes at top of zimbra.php file
- Added new SmartFolders feature - to allow users to control the list of
folders they wish to sync to their devices without using XML files
NOTE: If you wish to enable SmartFolders - you MUST stop sync and clear
all old state data from the state folder. Even if no SmartFolder names
are used, the sequence and naming of the folders will change
NOTE: If SmartFolders are enabled, XML files will not be processed
To enable the feature set ZIMBRA_SMART_FOLDERS to true in config.php
Once enabled, the final character in a folders name can take on special
meanings as follows :-
"-" Do not include this folder or any subfolders thereof
"." Include this folder - But do not include any subfolders thereof
- Changed Attachments handling to allow attachments in multi-part messages
to be returned. Also allows attaching of an email as a .eml file
- Added ability to filter out Completed Tasks during sync - Note: As zimbra
does not store a Completed Date, the last modified timestamp is used
- Added Reminders for Tasks (for zimbra 7.0+) - need a zcs 6.0.nn user to test
if it will work on zcs 6.0.nn also
- Enhancement in SendMail to include original message text for reply/forward
This is done in preference to attaching the original as Android devices are
unable to open .eml attachments natively at this time
- Added check in SendMail for z-push version to ensure we properly format the
path to the "state" directory - change is to match the config.php change to
the definition of STATE_DIR introduced by z-push 1.5
- Fix in SendMail to honour ZIMBRA_ENFORCE_VALID_EMAIL when set to true. If
the from email address used is not a valid one for the account it is
replaced by the default sender email address.
- Fix in SendMail to add sender name to the address in all cases. This is a
generalized fix based on the original premise of the ZIMBRA_NOKIA_MFE_FIX
as it seems that iPhone and Android also send emails without a sender name
- Fixed GetSearchResults to match response 'keys' to latest zpushdefs.php
- Fixed possible memory leaks and reduced memory usage by unset()ing rtf and
mime decoder class instances, and various arrays
- Removed the input parameter from the params array send to Mail_mimeDecode
as it is already passed to the class on creation, and created and unset a
params object to ensure that memory is cleaned up
- Fix for special characters (",&,<,>) in user passwords
- Added fix for Euro symbol & other non-standard cp1252 characters
- Changes made to MoveMessage to prevent items being moved from one account to
another, and from one folder type to another. These restrictions are to
prevent items becoming inaccessible on the device. See comments in function
for more information.
- Removed username from Sync Request log entry for z-push 1.5 and later as
the z-push engine is already logging it
- Fix for Categories field from HTC Desire (Android 2.2) - Phone sends an
empty string Categories field instead of an array. The blank field caused an
error in CreateTagRequest. Fix to check that (is_array(input->categories))
- Fix added for iPhone/iPad/etc Meeting Request/Response handling as it appears
that the Apple client sends off meeting requests/responses directly from the
handset instead of letting the server handle them
- Fix in PROVISIONING getPolicyKey to avoid writing "validate" records in
database for android devices
- Fix in PROVISIONING generatePolicyKey to limit upper bound on random number
generator to 2147483647 (as some phones cannot handle larger numbers)
- Added function ReportMemoryUsage (called on Logoff) to track memory use
- Added new functions that are needed for ActiveSync protocol version 12
- ImportMessageFlag - use of this required a change to request.php
- GetSettings - used for Out-Of-Office
- SetSettings - used for Out-Of-Office, and storing Handset data
- Updated GetSearchResults to add mailbox search capability
PLEASE READ THESE RELEASE NOTES COMPLETELY BEFORE IMPLEMENTING RELEASE 50
THIS IS EXPECIALLY CRITICAL IF YOU ARE TURNING ON ANY OF THE NEW FEATURES
Minimise use of Mail_MimeDecode - to avoid running out of memory
================================================================
Shortly after publishing Release 49 an issue was discovered when handling an
email with a 13MB attachment. The loading of the email, and processign it with
Mail_MimeDecode resulted in the session running out of memory. The session
crashed, and reported a memory allocation error in the Apache log file.
Prior to release 49, this function was only used once in the application - at
the point where the user is sending an email from the phone. It is used to
process the raw uploaded file to prepare it for zimbra to send.
Release 49 introduced it's use to other places, synching emails out to the
device, and for getting details of the original email for Smart Forward/Reply.
It was in these places where it was found to be consuming huge chunks of
memory whenever an email had large attachments. This could cause the sync to
crash. It would continue to crash every time the phone tried to sync the mail
folder until such time as the message was moved on the server, or was outside
of the window for days to sync.
Release 50 has replaced the functionality for which this function was used
with native zimbra soap calls.
Mail_MimeDecode is still needed for SendMail from the phone to handle the raw
rfc822 data sent to zimbra as it was in releases prior to 48 - hopefully
people will not be sending huge attachments from their phones but, even if
the send does fail, it will not affect the sync of emails out to the phone.
Fixed filtering Calendar/Contacts/Task virtual folders
======================================================
When ZIMBRA_VIRTUAL_<Folder type> is set to false for a folder type with
SmartFolders enabled, the folder list was not being filtered correctly. This
has been fixed, and should now be working as expected.
System-wide Disable Switches for Folder Types
=============================================
Several users have asked over time if it is possible to turn off Email sync or
Calendar sync for an entire installation. Up to now it has only been possible
through a hack to force all users to read a particular XML file.
This release makes it possible to turn off individual folder types through the
config file. Enabling any of these switches will turn off that folder type (for
example email) for every user of the system. It will prevent all users from
syncing their email to their mobile devices.
The default setting remains that these switches are either not present - or are
set to false - in which case all folder types will be available for synching.
=== To Disable Sync of any Folder Type System-wide ===
Add the appropriate definition from this block to config.php, and set it true
NOTE: Devices will need a full-resync to remove the unwanted content
define('ZIMBRA_DISABLE_MESSAGES',true);
define('ZIMBRA_DISABLE_CONTACTS',true);
define('ZIMBRA_DISABLE_APPOINTMENTS',true);
define('ZIMBRA_DISABLE_TASKS',true);
SmartFolders
============
WARNING TO SYSTEM ADMINISTRATORS. IF YOU WISH TO ENABLE SMART FOLDERS YOU MUST STOP
SYNC FOR ALL DEVICES, CLEAR OUT THE STATE DIRECTORY, AND THEN RESYNC ALL DEVICES
THIS IS DUE TO AN OVERHAUL IN THE NAMING AND SEQUENCING OF FOLDER IDs
For many releases now, there has been the ability to manipulate the number and types
of folders that could be synced for an individual user through the use of user.XML
configuration files. While this worked well, it meant that the system administrator
had to be involved in every change for every user.
This release introduces a new feature called SmartFolders as an alternative to XML
files that will hopefully make it easy for users who wish to manipulate their syncing
content to be able to do the job themselves.
Note that SmartFolders and XML files are mutually exclusive on the server. Enabling
SmartFolders disables the use of user.XML files for everyone.
To enable the feature set ZIMBRA_SMART_FOLDERS to true in config.php
Once enabled, the final character in a folders name can take on special meanings as
follows :-
"-" Do not include this folder or any subfolders thereof
"." Include this folder - But do not include any subfolders thereof
So for example you might have a top level folder called "Archive-" into which you
move all old folders that you do not want to be able to see from your device.
Or you might have a "ToBeFiled." folder that contains a number of child folders used
for longer term storage of reference emails. Naming it with a period (.) on the end
will allow you to see that folder on your phone - so you can move emails in there to
clear your Inbox - and then when you get to your desktop you can file all the emails
into the appropriate child folders.
The SmartFolders feature is particularly useful on Android devices - which list out
every folder from your server across the top of the email client.
=== To enable the Smart Folders feature ===
NOTE: Enabling SmartFolders will DISABLE user.XML processing
It will also rename/resequence folder ID's in the state file for
the device - so state directory must be cleared prior to it's use
define('ZIMBRA_SMART_FOLDERS',true);
User control of Folder types
----------------------------
NOTE: z-push does not do hierarchy resync properly at this time - so if a user wishes
to make changes to their sync rules using the following directives, they should first
remove the sync account from their phone - then make the changes - then re-add the
account to their phone.
When SmartFolders are configured, in instances where individual users wish to disable
particular folder types from syncing to their devices, a special top-level folder
structure can be used to configure those options. This provides the major filtering
capability that would have been available through user.XML files.
If a folder named '*SyncConfig*' is found, the system will use any child folders in
that folder to configure options.
The names of the sub-folders will be interpreted as configuration directives.
These should have the format of
<folder type>&<setting=value>[&<setting=value>& ...]
<folder type> can be any one of message, contact, appointment, task and
<setting=value> can be any one of
active=true/false - active=false will disable sync of that folder type
virtual=true/false - virtual=false will turn of sync of additional folders (not
applicable for message type)
primary=FolderName - setting primary will override the default primary folder for
that content type (Inbox, Contacts, Calendar, Tasks)
So, for example, you could have
*SyncConfig*
message&active=false to disable email sync
task&active=true&virtual=false to limit the task sync to the primary folder
appointment&active=true&virtual=false&primary=WorkCalendar to limit
appointment sync to the WorkCalendar folder
The *SuncConfig* folder, and it's contents will never be synced to the phone so
long as ZIMBRA_SMART_FOLDERS is set to true.
Improvements in Attachment Handling
===================================
Attachments handling has been completely re-written to allow attachments in multi-part
messages to be returned. This restructuring also has an added benefit in that it
allows attaching of an email as a .eml file to another email.
Note: Android is currently unable to open attached email messages - but other clients
should be able to do so.
Improvements in Task Sync
=========================
Added ability to filter out Completed Tasks during sync - Note: As zimbra does not
store a Completed Date, the last modified timestamp is used
Added Reminders for Tasks (for zimbra 7.0+) - I no longer have a zcs 6.0.nn install
so if someone running version 6 can uncomment the 7.0 restriction and test it would
be very much appreciated.
Improvements in SendMail
========================
The entire SendMail function has been re-written to decompose the mime message
received from the phone, and to recompose it prior to sending. This allows for
easy fixing of the from address, as well as implementing SmartForward and
SmartReply (where the phone does not send the original content with the reply/
forward text)
The major enhancement in SendMail is to include original message text for reply/
forwarded emails. This is done in preference to attaching the original as Android
devices are unable to open .eml attachments natively at this time. In a future
release I may look to make this optional - or to follow the configuration in the
users Web client preferences
Added check in SendMail for z-push version to ensure we properly format the path
to the "state" directory - change is to match the config.php change to the
definition of STATE_DIR introduced by z-push 1.5 - The emails to be send are first
uploaded to the state directory, and then send from there by zimbra.
Fix in SendMail to honour ZIMBRA_ENFORCE_VALID_EMAIL when set to true. If the
from email address used is not a valid one for the account it is replaced by the
default sender email address.
Fix in SendMail to add sender name to the address in all cases. This is a
generalized fix based on the original premise of the ZIMBRA_NOKIA_MFE_FIX as it
seems that iPhone and Android also send emails without a sender name
Search should now return proper results
=======================================
Fixed GetSearchResults to match response 'keys' to latest zpushdefs.php
Company searches should now return lists of users as expected.
Improvements in Memory Management
=================================
Fixed possible memory leaks and reduced memory usage by unset()ing rtf and mime
decoder class instances, and various arrays
Note: even with these improvements, the memory footprint of an individual session
has increased slightly as a result of all the new functionality
Removed the input parameter from the params array sent to Mail_mimeDecode as it
is already passed to the class on creation, and created and unset a params object
to ensure that memory is cleaned up
Fix for Special Characters in Passwords
=======================================
Fix for special characters (",&,<,>) in user passwords. These characters should
no longer prevent users from synching.
Fix for Euro Symbol and Other cp1252 Characters
===============================================
Added fix for Euro symbol & other non-standard cp1252 characters that are not
represented in UTF-8
Restricted MoveMessage to single accounts/folder types
======================================================
Changes made to MoveMessage to prevent items being moved from one account to
another, and from one folder type to another. These restrictions are to
prevent items becoming inaccessible on the device.
It seems that when you move a message from a folder shared with you into one of
your own folders, ór vide-versa, zimbra assigns it a completely new message id.
However, there is no way to retrive what that new message id is through the SOAP
call. Therefore we ended up with a message now in the receiving folder that the
phone thinks still has it's original id from the originating folder. However, on
the server the id has changed. The item id on the phone therefore does not exist
on the server - and as a result cannot be moved/deleted/manipulated on the phone.
It will just sit there until the sync window (3 days for example) causes it to
drop out of the phone.
To avoid this situation - the phone is now prevented from moving items from one
users folder to another user's folder. Any attempt to move an item in this way
will return an error to the phone, and will log a message to the debug.txt file.
Tidied up SyncRequest log entries
=================================
Removed username from Sync Request log entry for z-push 1.5 and later as the
z-push engine is already logging it
HTC Android - Categories Fix
============================
Implemented a fix for Categories field from HTC Desire (Android 2.2)
In the case where a modified item on the phone, has no categories associated
with it, the phone sends an empty string in the Categories field. This is unexpected
in two ways. Firstly, most phones simply omit the field as it is optional. Secondly,
it sends a string variable instead of an array of string variables. The blank
field caused an error in CreateTagRequest.
The fix is to check that (is_array(input->categories)) prior to calling the
CategoriesToTags function.
Apple iPhone/iPad Meeting Request/Response Fix
==============================================
Note: As I do not own or have access to any Apple devices, I have been limited to
third party testing on this. It may still need more work.
A fix has been added for iPhone/iPad/etc Meeting Request/Response handling as it
appears that the Apple client sends off meeting requests/responses directly from the
handset instead of letting the server handle them. This was resulting in duplicated
emails going to and from the attendees and meeting organizer.
Hopefully the behaviour will be much better with these changes.
Improvements in PROVISIONING handling
=====================================
Fix in PROVISIONING getPolicyKey to avoid writing "validate" records in
database for android devices
Fix in PROVISIONING generatePolicyKey to limit upper bound on random number
generator to 2147483647 (as some phones cannot handle larger numbers)
Added Memory Use Reporting Function
===================================
Added function ReportMemoryUsage (called on Logoff) to track memory use
Added new Functions needed for ActiveSync 12 protocol
=====================================================
Added new functions that are needed for ActiveSync protocol version 12
- ImportMessageFlag - use of this required a change to request.php
- GetSettings - used for Out-Of-Office
- SetSettings - used for Out-Of-Office, and storing Handset data
- Updated GetSearchResults to add mailbox search capability
============== Z-Push Zimbra Backend - Release Notes - Revision 50 ==============
======================== CHANGES MADE IN EARLIER RELEASES ========================
Changes Made To Revision 48:
- Critical fix for v5 compatability fix. LookupV5Timezone missing the
$this-> pointer, so calls to the function caused crash in backend.
- Fix typo in name of v5timezone.xml file
- Added more error handling/logging around zimbra 5 compatability code, and
updated the v5 compatability notes
- Change to ExtractSessionID to return session for zimbra 5
- Change AlterPing to always return false for zimbra 5 as i4ms flag used by
AlterPingChanges does not exist on zimbra 5. Old folder ping method must
be used
- Fix in GetZimbraFolders to check for a folder identifier (l) before adding
the folder to the folder list. This is to avoid including any folders from
the notify block in the soap response
- Moved setting of _wasteID out of GetFolder and into Setup as GetFolder is
only called on a Hierarchy Sync - not on every sync - so it ended up being
set to false more often than not
- Fix in GetMessage to not create another meeting request out of a meeting
acceptance email
- Fix in CreateMeetingRequest to add invitees to the meeting if they are
provided, and to send out meeting invitations to the email addresses
specified (Android sends them - Nokia/WM don't - Not sure about iPhone)
- In function GetBody added html_entity_decode and preg_replace for duplicate
blank lines to make plain text output more readable when there is no
text/plain body part in the message
- Moved PROVISIONING: getDeviceRWStatus log message so it only logs for
unrecognised devices or those that have had a device wipe requested
- Added option to define ZIMBRA_DEBUG as 'user1' or 'user1,user2,userN' to
enable zimbra soap debug messages for just the listed accounts. Defining
it as true/false will continue to work as before
Changes Made To Revision 47:
- Added NokiaFakeOut to allow Nokia E72 and some other models to Send emails
- Added compatability fix for zimbra 5 timezone handling (see notes below)
- Changed ConvertItemIdToInvID to check for shared item IDs (i.e. look for a
":" in the passed $id) in case user modifies a shared appt on phone
- removed l= specified from ModifyAppointmentRequest to avoid moving a shared
appt to user's own Calendar when editing it on the phone
- Fix to avoid replacing the organizer name/email in ModifyAppointmentRequest
- Check for Organizer Name exists before outputting (to avoid error in log)
- Fix in GetBodyRecursive to use mb_detect_encoding() to identify the original
encoding of an email - prior to converting it to UTF-8 for the phone. To use
mb_detect_encoding, the PHP extension package php-mbstring must be installed
Changes Made To Revision 46:
- Added new functions TagsToCategories & CategoriesToTags to support syncing
Categories<->Tags on all content types
- Extended Categories<->Tags syncing to Appointments/Tasks (Note: z-push doesn't
support categories on Message sync in z-push 1.4)
- Fixed loop counter in Setup function to prevent recursion through all newly
added folders when checking for children of linked folders
- In StatMessage function for 'message' replaced call to GetMsgRequest with call
to GetMsgMetadataRequest to reduce data transfer and logging
- Improved sync of Birthday/Anniversary - Dates on zimbra must be in YYYY-MM-DD
format (or if --MM-DD format this/next year will be added depending on date)
- Fixed Children sync on Contacts by treating the value as an array instead of
a string. Uses implode/explode (Note: does not sync to Nokia Children field)
- Hard linked webpage sync on Contacts to workURL on zimbra as only one of the 3
workURL, homeURL and otherURL can be synced out to the phone, and once sync'ed
if a change is made to the contact on the phone, we would not know which of the
three URL fields to sync webpage to on an incoming contact
- Fix to avoid overflowing 32-bit integer for calExpandInstStart and
calExpandInstEnd when syncing calendar/tasks. Use strval() and append "000"
instead of multiplying by 1000
- Replaced divide by 1000 with substr() to avoid conversion issues in
Date4ActiveSync and GetMessage (appointment request email)
- Added new CONFIG.PHP key ZIMBRA_DEBUG to enable/disable logging of the SOAP
requests & responses, and the Setup Folder Lists. This allows more granular
debugging than using on the value of the WBXML.PHP key WBXML_DEBUG that turns on
Z-Push internal logging. It is now possible to log either on their own, or both
- Fix in GetMessage to avoid e_attribute_?? arrays to match names with email
addresses, as a blank value for a persons name would result in the incorrect
Personal name being matched with any subsequent email addresses
- Removed unused code and debug statements, and tidied up code
- Added DeviceType on new, and updated DeviceType and UserAgent on sync in
getDeviceRWStatus to show most recent client version that sync'ed
- Fixed reference to this_WasteID instead of $this->_wasteID
Changes Made To Revision 45:
- Implemented the provisioning "Remote Device Wipe" functionality. This needs a
MYSQL database to work. To disable it, set PROVISIONING to false in config.php
- Sync Device Categories with Server Tags (requirement for zimbra 7)
- Removed html="1" from GetMsgRequest calls in StatMessage to reduce data returned
- Fixed getBody to compare on passed $html variable instead of $this->_useHTML
- Added comments & Reduced logging in AlterPingChanges
Changes Made To Revision 44:
- Fixed parameters on iconv call added by Release 43 to handle foreign languages
Changes Made To Revision 43:
- Implemented the new AlterPing/AlterPingChanges method from 1.4 to reduce the
number of hits on the backend for monitored folders. Operates on the basis of
monitoring the i4ms flag on folders for changes
- Added handling of RTF field in appointment/task/contact where no Body/Notes
field received. Requires z_RTF.php include file from as12 branch of z-push SVN
to decode the Compressed RTF stream. Issue reported by Danish language user
- Removed multiple classes/functions that were duplicates of those in diffbackend
to simplify the updating of the code to match future z-push releases. We should
only declare "standard" classes/functions where we have a need to override.
- Changed debug logging of Soap Request/Response to follow WBXML.PHP debug flag
so that basic application flow debugging will not flood logs
- Changed date selection criteria for Appointments/Tasks to use the attributes
calExpandInstStart and calExpandInstEnd where Start is determined by cutoffdate
or defaulted to 366 days in the past and End is arbitrarily set to 366 days in
the future
- Fixed some issues with recurring appointments on last day or named day of month
Outgoing appointments would set WeekOfMonth to -1 when ActiveSync wants a value 5
Incoming appointments would set bysetpos to 5 where zimbra wants a value -1
- Changed default for all folder types to recursive=true - XML will still override
Changes Made To Revision 42:
- Fixed typo in $this->_virtualAddressbook missing an "s" on end
- Added isset checking to clean up "undefined index" entries in debug log
Changes Made To Revision 41:
- Fixed Bug With SendAsEmail and SendAsName
Changes Made To Revision 40:
- Added 'function_exists' Check for Timezone Functions
- Fixed Some Profile Parsing Issues
- Fixed Bug Where '&' in Subject or Body of Appointment Would Cause it Not to Sync
Changes Made To Revision 39:
- Removed PHP-SOAP From Requirements
- Added Error Checking For PHP-CURL
- Changed Tag For Search Folders In User File
- Meeting Organizer Name & Email Uses SendAsName and SendAsEmail From User File. If Not Specified, Get From Zimbra
- Added Config File Option (ZIMBRA_ENFORCE_VALID_EMAIL) To Prevent User (True) From Using Email Address (SendAsEmail) Not Owned By Them
Changes Made To Revision 38:
- Search Folder Support Added For Mail
- Fixed Bug In Profile Loading
- Fixed Bug That Was Not Handling The Absense Of The "Optional" AllDayEvent Flag From The Phone (Palm Pre Issue)
- Fixed Bug In AllDay Appointments Timezone
- Fixed Bug Where Multi-Day AllDay Server Appointment Appearing As Single Day On Phone
- Fixed Bug In Generating Timezone Blob From Zimbra Appointment That Caused Recurring Appointments To Move
- Fixed Meeting Request Emails So They Request Response (Accept/Decline/Tentative)
- Implemented MeetingResponse To Handle Meeting Requests (Requires Z-Push 1.4 Or Later)
- Refactored To Use $tzObject To Identify Every Instance Of A Timezone Identifier For Converting To/From ActiveSync
(Excessive Use Of $tz Was Getting Confusing)
- Removed Some Nokia Specific Code That Is No Longer Needed Now Timezones Are Working Better
Changes Made To Revision 37:
- NOTE TAG CHANGES TO USER PREFERENCE FILES
- Added Ability For Multiple Profiles In User Preference File (See Below For Details)
- Fixed Timezone Handling For Calendar Appointments
- Added Ability To Specify Roaming Timezone (See Below For Details)
- Tasks Implemented - Note: Zimbra supports a limited set of the ActiveSync Task functionality
(eg. Date Completed/Reminders/Repetition/Categories are not supported)
Credits
=======
This php script makes use of mime.php and mimePart.php from the PEAR project
Download Latest Version
zimbra73.tgz (224.7 kB)
