First of all, sorry if this post doesn't go here, I couldn't find a more suitable place. I also couldn't find any solution to this problem in this forum, I hope it is not a duplicated post.
I have been working with the Mail to ticket automation extension, in its 3.2.0 version. I think I found a way to get rid of all the message replies when replying to an iTop notification from Outlook. In the emailmessage.class.inc.php class found in /combodo-mail-to-ticket-automation/combodo-email-synchro/classes/ we go to the GetNewPartHTML method and in the $aTagsToRemove array we add a couple things so it looks like the image that I attached.
I have been testing it for a couple of days now and I didn't find any problems in the Outlook or Gmail responses. I found this tags poking with the .eml file in the Incoming eMail Inboxes window, maybe you can find a better way to avoid the Outlook replies.
I cannot find any div with the divRplyFwdMsg attribute in our emails sample... Which Outlook version was used ?
Also, prefer using the configuration instead of overwritting the module code : with the later you'll lose your customization the next time you update ! This can be configured in the html-tags-to-remove module config parameter. See extension documentation
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is the part where I found the tag, I have changed some content of the message and the email addresses for privacy reasons, but all the tags are untouched:
As you can see, there is a div present with id=3D"divRplyFwdMsg", and that is the one that contains the previous message. I sent the message from the browser application (outlook.live.com), so I still have to test it with the desktop application to know if the tag is also present when the message is sent that way.
Also, I will move the tags to the configuration file, thanks for the heads up.
Happy Holidays, Regards.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'd like to ask if many users encountered this type of div ? If yes, then it might be a good idea to add it in the option default value ?
Maybe @Jeffrey you've got an opinion on this ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I haven't spotted this div yet, but I also haven't investigated it since most of the emails came from the same org with the same email client.
I'm not a fond of blindly stripping HTML elements from an email (I explained this in https://github.com/Combodo/combodo-email-synchro/pull/4 ; basically someone might reply or forward sth but there could be new inline info added. Another example: I often forwarded mails sent to me personally to our helpdesk.)
The logic would need to be more thorough and compare to existing log entries. For which I have a whole other motivation that they could use a revision so each log entry is a separate DB record :) )
Last edit: Jeffrey Bostoen 2021-12-30
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, in fact we have one of our biggest clients using an old and less secure application mainly because they use Outlook and don't like the clutter that is created in the tickets when they reply from their emails. The whole thread is pasted into the public log including signatures and it's a big mess. If we can fix this, then we can move on to using only iTop.
Just one more thing that I wanna point out, the quote part of the email isn't erased by only adding the divRplyFwdMsg div, it's a combination of that div and other HTML tags, such as "p" or "br". In my tests with Outlook and Gmail I didn't encounter any issues when adding these tags to the tagsToRemove array, but that is only our use case and should be tested thouroughly before adding it as a default value.
If you guys need me to test anything beforehand, feel free to hit me up.
Regards.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, in fact we have one of our biggest clients using an old and less secure application mainly because they use Outlook and don't like the clutter that is created in the tickets when they reply from their emails.
Do you mean your client is now using another ticketing system, and you're testing iTop as a replacement ? (I don't think I misunderstand but just checking...)
Just one more thing that I wanna point out, the quote part of the email isn't erased by only adding the divRplyFwdMsg div, it's a combination of that div and other HTML tags, such as "p" or "br". In my tests with Outlook and Gmail I didn't encounter any issues when adding these tags to the tagsToRemove array, but that is only our use case and should be tested thouroughly before adding it as a default value.
So modifying the configuration didn't worked ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Do you mean your client is now using another ticketing system, and you're testing iTop as a replacement ? (I don't think I misunderstand but just checking...)
Yes, they are using old software that doesn't have this reply from email functionality. We use iTop for a lot of things, it's great software, so we are trying to push this client towards iTop and ditch the older software.
So modifying the configuration didn't worked ?
It did work for us, I'm just saying that adding that div doesn't make the thread dissapear, it's the combination of that div and other html tags. As Jeffrey said, it may not be the best solution because you are just ignoring common tags, and even though I tested with a lot of different email structures, there may be a case where someone pastes something with a "p" tag and it gets deleted by the process. It works for us, but I think it needs more polish if you intend to put it in the default array. I meant this thread as a temporary solution and to spark a discussion about this problem, because I have read some threads about this topic but nobody has any solution to it.
As I read in an older thread about this topic, the best solution may be to stop using Outlook, but that decision is not ours to make.
Let me know if I can help with anything else,
Regards.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks Pierre! I know Lars (ITOMIG) also suggested this once.
It would allow for easier processing, such as for instance individual editing; sorting; updating info (change author for instance or fix timestamp); easier comparison (check if message is the same or if a big part is the same); ...
Another use case was that if this was an object class in the datamodel, it could easily be extended; for instance with a third party ID (for instance ID of a message on Twitter, Slack, GitHub, WhatsApp, email ID, ... ) or other info.
Last edit: Jeffrey Bostoen 2022-01-05
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the summary, I wasn't sure to recall all of what was said before :/ I began searching the forum but wasn't able to find old discussions on this subject... Do you have any link to messages where this was discussed ?
For me the biggest benefit of saving caselog entries as individual objects is querying.
But a big risk is impacting performance, and also migration will be of course a big question...
Everything else (all of what you mentionned I think) could be addressed with new iTop core methods (in ormCaseLog I guess ?), right ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think most of it came up in the unofficial Slack channel.
Performance would indeed be the big question. Some stuff could indeed be fixed with additional methods; but extending would be more difficult. Although it would be a good idea if there's at least one "third party ID" field for all sorts of integrations. Worst case, multiple strings with info could be stored as one.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When defining log entries as objects in the datamodel, it become a lot more easier to extend. Additional to those benefits which was already mentioned I am thinking for instance about more context information of incoming mails:
Let's imagine we've got a class called "CaseLogEntry" which got attributes like "content (HTML)", "linked object", "timestamp", ...
It would be possible to extend and create a sub class "CaseLogEntryFromMail" and include attributes like "Full mail Text" (not just the new part), "Email-Header", ...
I could image to store LogEntries in a structure a little bit like CMDBChangeOp.
It may be that all of those features also could be covered by new iTop core methods, but that would be less flexible and it would be more or less not possible to do it via an extension (i.e. to cover special use cases)
Anyhow: You are right Pierre it may be not that easy due to the performance and and the migration. (But correct me if I am wrong you already wrote such migrations in the past for instance in the module request-templates-base?)
Hello,
First of all, sorry if this post doesn't go here, I couldn't find a more suitable place. I also couldn't find any solution to this problem in this forum, I hope it is not a duplicated post.
I have been working with the Mail to ticket automation extension, in its 3.2.0 version. I think I found a way to get rid of all the message replies when replying to an iTop notification from Outlook. In the emailmessage.class.inc.php class found in /combodo-mail-to-ticket-automation/combodo-email-synchro/classes/ we go to the GetNewPartHTML method and in the $aTagsToRemove array we add a couple things so it looks like the image that I attached.
I have been testing it for a couple of days now and I didn't find any problems in the Outlook or Gmail responses. I found this tags poking with the .eml file in the Incoming eMail Inboxes window, maybe you can find a better way to avoid the Outlook replies.
Regards.
Hello,
Just seeing your message now sorry.
I cannot find any div with the divRplyFwdMsg attribute in our emails sample... Which Outlook version was used ?
Also, prefer using the configuration instead of overwritting the module code : with the later you'll lose your customization the next time you update ! This can be configured in the
html-tags-to-removemodule config parameter. See extension documentationHello Pierre,
This is the part where I found the tag, I have changed some content of the message and the email addresses for privacy reasons, but all the tags are untouched:
As you can see, there is a div present with id=3D"divRplyFwdMsg", and that is the one that contains the previous message. I sent the message from the browser application (outlook.live.com), so I still have to test it with the desktop application to know if the tag is also present when the message is sent that way.
Also, I will move the tags to the configuration file, thanks for the heads up.
Happy Holidays, Regards.
Thanks !
I'd like to ask if many users encountered this type of div ? If yes, then it might be a good idea to add it in the option default value ?
Maybe @Jeffrey you've got an opinion on this ?
I haven't spotted this div yet, but I also haven't investigated it since most of the emails came from the same org with the same email client.
I'm not a fond of blindly stripping HTML elements from an email (I explained this in https://github.com/Combodo/combodo-email-synchro/pull/4 ; basically someone might reply or forward sth but there could be new inline info added. Another example: I often forwarded mails sent to me personally to our helpdesk.)
The logic would need to be more thorough and compare to existing log entries. For which I have a whole other motivation that they could use a revision so each log entry is a separate DB record :) )
Last edit: Jeffrey Bostoen 2021-12-30
Thanks for your feedback !
Hello Pierre,
Yes, in fact we have one of our biggest clients using an old and less secure application mainly because they use Outlook and don't like the clutter that is created in the tickets when they reply from their emails. The whole thread is pasted into the public log including signatures and it's a big mess. If we can fix this, then we can move on to using only iTop.
Just one more thing that I wanna point out, the quote part of the email isn't erased by only adding the divRplyFwdMsg div, it's a combination of that div and other HTML tags, such as "p" or "br". In my tests with Outlook and Gmail I didn't encounter any issues when adding these tags to the tagsToRemove array, but that is only our use case and should be tested thouroughly before adding it as a default value.
If you guys need me to test anything beforehand, feel free to hit me up.
Regards.
Hello,
Do you mean your client is now using another ticketing system, and you're testing iTop as a replacement ? (I don't think I misunderstand but just checking...)
So modifying the configuration didn't worked ?
Hello,
Yes, they are using old software that doesn't have this reply from email functionality. We use iTop for a lot of things, it's great software, so we are trying to push this client towards iTop and ditch the older software.
It did work for us, I'm just saying that adding that div doesn't make the thread dissapear, it's the combination of that div and other html tags. As Jeffrey said, it may not be the best solution because you are just ignoring common tags, and even though I tested with a lot of different email structures, there may be a case where someone pastes something with a "p" tag and it gets deleted by the process. It works for us, but I think it needs more polish if you intend to put it in the default array. I meant this thread as a temporary solution and to spark a discussion about this problem, because I have read some threads about this topic but nobody has any solution to it.
As I read in an older thread about this topic, the best solution may be to stop using Outlook, but that decision is not ours to make.
Let me know if I can help with anything else,
Regards.
Hello,
Ok, thanks for all those additional info !
Hello,
Thanks for the feedback.
Indeed analyzing mail content is not easy.
If anyone has better solutions, we are open to contributions :)
@pgoiffon is there any thoughts on restructuring how log entries are stored?
This would make this a lot easier.
Anyhow, if I have some time or of someone has an incentive to prioritize this project, I might look into some mechanic using the existing log format.
Hello,
I've forwarded your message to the product team.
We have no plan for now to change the way caselogs are stored - only a ticket (N°1365) to allow to edit them individually.
Thanks Pierre! I know Lars (ITOMIG) also suggested this once.
It would allow for easier processing, such as for instance individual editing; sorting; updating info (change author for instance or fix timestamp); easier comparison (check if message is the same or if a big part is the same); ...
Another use case was that if this was an object class in the datamodel, it could easily be extended; for instance with a third party ID (for instance ID of a message on Twitter, Slack, GitHub, WhatsApp, email ID, ... ) or other info.
Last edit: Jeffrey Bostoen 2022-01-05
Hello,
Thanks for the summary, I wasn't sure to recall all of what was said before :/ I began searching the forum but wasn't able to find old discussions on this subject... Do you have any link to messages where this was discussed ?
For me the biggest benefit of saving caselog entries as individual objects is querying.
But a big risk is impacting performance, and also migration will be of course a big question...
Everything else (all of what you mentionned I think) could be addressed with new iTop core methods (in ormCaseLog I guess ?), right ?
I think most of it came up in the unofficial Slack channel.
Performance would indeed be the big question. Some stuff could indeed be fixed with additional methods; but extending would be more difficult. Although it would be a good idea if there's at least one "third party ID" field for all sorts of integrations. Worst case, multiple strings with info could be stored as one.
Hi there,
wish you all the best for 2022!
Basically I agree with Jeffery.
When defining log entries as objects in the datamodel, it become a lot more easier to extend. Additional to those benefits which was already mentioned I am thinking for instance about more context information of incoming mails:
Let's imagine we've got a class called "CaseLogEntry" which got attributes like "content (HTML)", "linked object", "timestamp", ...
It would be possible to extend and create a sub class "CaseLogEntryFromMail" and include attributes like "Full mail Text" (not just the new part), "Email-Header", ...
I could image to store LogEntries in a structure a little bit like CMDBChangeOp.
It may be that all of those features also could be covered by new iTop core methods, but that would be less flexible and it would be more or less not possible to do it via an extension (i.e. to cover special use cases)
Anyhow: You are right Pierre it may be not that easy due to the performance and and the migration. (But correct me if I am wrong you already wrote such migrations in the past for instance in the module request-templates-base?)
We also already discussed that topic with @cisou during a call. There was also a brief discussion on that topic here: https://sourceforge.net/p/itop/tickets/1972/
Hello,
Many thanks to both of you for the additional info.
I've added a new Bug object in our internal DB so that this modification can be discussed : N°4709.
Another use case (based on a different ticket system): it would be more easy to create a new ticket from a new log entry.
Use case: existing ticket with question A; customer suddenly asks info about ticket B. => split case log entry in its own ticket.
And it seems there was a ticket recently from someone else with another use case: https://sourceforge.net/p/itop/tickets/1483/?limit=25#24ab
@scheibling what do you think ?
I think you meant @larhip ?
Wooops yes, my bad ! thanks :)