Menu

How to instruct FP to treat content as plain text

macmarrum
2021-12-04
2021-12-07
  • macmarrum

    macmarrum - 2021-12-04

    I want to enter literal HTML code in a node.

    In other words, I want a plain-text node content which will contain the following:

    <html>
        <head>
            <style>.my-class { border: 1px solid blue; }</style>
        </head>
        <body>
            <div class="my-class">
                Some text in a div
            </div>
        </body>
    </html>
    

    I don't want FP to interpret it as HTML and display the interpreted result (i.e. to display "Some text in a div" in a blue frame).
    I know that I can achieve it by pasting the content into SimplyHTML (converting it to HTML representation of plain text) or that I can enter it as a formula. I want to avoid such workarounds, if I can.

    At first, the task seemed simple. I set Core text->Format = Text. But it did not work.
    I consulted @dpolivaev, who made me aware that Format == Text doesn't mean use text/plain for the content. It rather means: do not evaluate any formula, otherwise, if the content looks like HTML or has CSS applied, interpret it as HTML.

    Interestingly, Format == Standard means the same as Format == Text but first any formula is evaluated.

    Example

    Given the following content is entered as Core Text:

    ='''# Heading 1
    Paragraph'''
    

    When Core text->Format == Text, FP displays the content literally.
    When Core text->Format == Standard (and provided formula evaluation is on), FP evaluates the code as a Groovy formula.
    When Core text->Format == Markdown (and provided formula evaluation is on), FP converts the Markdown resulting from the Groovy-formula evaluation into HTML and displays it.

    Feature Request

    My idea is to change the functionality of Format == Text to mean "always treat the content as plain text, never interpret it as HTML or LaTeX or formula or date or number".

    Building on top of this, if a node had Format == Text, CSS would be ignored.
    The current implementation applies CSS to node content even when it's plain text (i.e. event if the text has no formatting of individual words, it's not HTML, and it's not Markdown).

    Note: CSS functionality was added to 1.9.12-pre03 -- see also
    Custom CSS style support in Preview 1.9.12_03
    How to improve Markdown in Freeplane?

    Example

    I want each Markdown (and HTML) paragraph to be blue, like so

    p {
      background: #E0F0FF;
      border-left: 5px solid #55AAFF;
      margin-left: 2px;
      padding: 5px;
    }
    

    I can do that by adding the CSS to Default, so that it covers each node (unless overriden by CSS with another Style applied to a node or by CSS added individually to the node). Once I add the above CSS to Default, text of each node (also plain text) has the formatting defined in the CSS. My expectation is that only HTML/Markdown content is styled by CSS and the other content is formatted using the usual means.

    My idea to get where I want is to mark plain-text nodes as Format = Text, instructing FP to ignore any CSS for them.

    I am curious to learn what the FP community thinks.

     
  • Dimitry Polivaev

    Actually, this issue is different.
    If you enter text starting with <html> or even <table> into editor, Freeplane treats it as formatted html. This behavior was originally defined in FreeMind and remained unchanged until now. Actually I think it is wrong because now there are other ways to create html content and I can remove this logic.

    Any disagreements?

     
  • macmarrum

    macmarrum - 2021-12-05

    @dpolivaev thanks for your proposal. The way I understand it, the new code (regexp) you presented prevents FP from interpreting anything with <html> or <table> as HTML when writing/pasting in the in-line editor. Still, the content is stored as rich text (HTML). In other words, the result is the same as if I opened SimplyHTML and pasted my HTML code in the Layout view.

    My need/requirement is different.

    I think that your proposed change might make FP more consistent (similar behavior in in-line and in SimplyHTML editors), still the issue of storing HTML code (or any content) as plain text remains to be solved.

    I am open to discussing my feature request: to change the functionality of Format == Text to mean "always treat the content as plain text, never convert it to rich text, never interpret it as HTML or LaTeX or formula or date or number".

     
  • Dimitry Polivaev

    At the moment when text is saved in a text editor Freeplane checks if it contains formatting.
    Here <html> gets some special treatment. If I worked on this feature further I would need to add an internal flag to enforce treating and saving html as plain text.

    However I do not understand why this issue is so important.
    I do not think that many Freeplane users have this problem.
    Notice that it you put a space ahead of your html code it would not be treated as html and saved as plain text.

    If the recent workaround does not help you I can revert it, please let me know.

     
  • macmarrum

    macmarrum - 2021-12-05

    If the recent workaround does not help you I can revert it, please let me know.

    The regexp change does not help me. Please feel free to revert it.

    Notice that it you put a space ahead of your html code it would not be treated as html and saved as plain text.

    Using an extra space does not really help (it is stripped at some point). But your comment gave me an idea. Using another character does help. Or even a string, like <!DOCTYPE html>.

    I can live with the workaround.
    Thank you for your help.

     
  • Milavon

    Milavon - 2021-12-05

    I tottally agree with macmarrum, the SimplyHTML should treat plain text as text, wether in loading or saving.

    for me , I usually save a lot of text in node notes, (rarely using formatted text or HTML)
    using the note dialog editor (SimplyHTML) to input the text, which sometimes can be large amount of text, for example (2MB)
    and altough I always set the note content format as (Text), it is very painful to load this text in
    the note dialog editor (would be very slow heavy, when the text size > 500KB)
    I can only get around these diffulties by using an input script (to store the text directly in the note without having to input the text from the note dialog editor) and using an output script
    to get the text from the note to the clipboard, then paste it into an external text editor to view / edit it

    and I think that the reason for this performance problem in (SimplyHTML), is that Freeplane contintues to save the plain text content as HTML content in FP XML file, for example:

    <richcontent TYPE="NOTE" CONTENT-TYPE="xml/html">
    <html>
      <head>
    
      </head>
      <body>
        <p>
     .........THE PLAIN TEXT CONTENT STORED AT THE NOTE.......
        </p>
      </body>
    </html></richcontent>
    

    and what make things more complicated: it has to replaces each line break in the plain text content with the HTML tags:

        </p>
        <p>
    

    and also replaces alot of whitespaces with the tags:

    &nbsp;
    

    all of these unnecessary processings take more time & increase the size of Freeplane file
    because every time the user load a plain text content,
    all these conversions are made before FP load the plain text for the user
    and also every time the user save a plain text content, the same conversions happen in the opposite direction. (some plain text characters converted to their HTML counterparts)

    I can understand the necessary conversions like escaping the XML characters (& " ' < >)

    I think that the plain text should always be treated as a plain text,
    and saved as a plain text in Freeplane file and then parsed directly as a plain text.
    this would make things very easier and make SimplyHTML more light & faster. specially for the users who mostly use the plain text and rarely use other formats (HTML, markdown, formulas,..)

    I used to think that the plain text cannot be stored in XML file naturallly until I saw an example
    (of how an XML file can store plain text easily without having to convert them to HTML tags)
    here is an XML file (exported from Keepass database application):

    <String>
                            <Key>Notes</Key>
                            <Value>Appointment Reminders
    
    Three out of ten people cant tell you what day of the week it is, let alone remember any of their appointments, so you have to give them a friendly nudge. Customers will definitely appreciate being given a heads up about their appointment.
    
    Below are a few tips for sending effective appointment reminder text messages:
    
        Make sure to include the date, time, location, your name (or your companys name), and what the appointment is for.
        Avoid the use of capital letters as it can make your message sound aggressive.
        Address your client by their name to make it sound more friendly.
        If you have a very busy schedule, advise your client to arrive 5-10 minutes early.
        If you established a no-show policy, make sure to inform your client in the reminder message.
    </Value>
                        </String>
    

    in this example, everything is clearn & natural & readable
    the line breaks, the spaces conversion

    So, I hope if it possible to change the way Freeplane & SimplyHTML treats plain text and just treat it as it is

    Regards

     

    Last edit: Milavon 2021-12-05
  • Dimitry Polivaev

    You both make some assumptions about Freeplane behavior and its reasons. Let me try to explain here how it actually works and why.

    Status quo

    Freeplane xml content

    Currently plain text content including formulas is saved as attribute inside xml tag <node>. Using a new element for it would allow to have line breaks, but it would break compatibility with old FreeMind and Freeplane versions. It means that after the change neither FreeMind nor any older Freeplane versions would be able to open mind maps saved with the newer version. It would also affect all existing xslt transformations used for export, they would have to be adjusted. This is the reason why I never replaced it by a new element with text.

    Editing

    Node core, detail and note editors always work with html representation. If needed it is created on the fly from plain text representation when editing starts. Because editing is always restricted to one node content the conversion can be performed fast enough.

    After the editing details and notes are always saved as (x)html. The core is saved as HTML only if Freeplane recognizes that html contains formatting.

    This decision is made by following method applied to html content created by editor:

                Pattern FORMATTING_PATTERN = Pattern.compile("<(?!/|html>|head|body|p/?>|!--|style type=\"text/css\">)|&lt;(?:html|table)&gt;",
                    Pattern.CASE_INSENSITIVE);
    
    private boolean containsFormatting(final String html) {
            final Matcher matcher = FORMATTING_PATTERN.matcher(html);
            return matcher.find();
        }
    

    It means that if you start your text with <!DOCTYPE html> the current heuristics would assume that it contains formatted html and treat it as html, not as plain text.

    Viewing

    Node content goes throw a chain of transformations responsible for evaluation of formulas, numbering, formatting. If the result starts with <html> or CSS is active for current node it is rendered as html (plain text gets converted to html if needed), otherwise it is rendered as plain text.

    Possible changes

    • Freeplane could save plain text in xml element instead of attribute breaking compatibility with FreeMind and older versions of Freeplane,
    • Freeplane could have a new explicit flag indicating "text/plain" so that even text starting with <html> could be displayed as plain text,
    • Freeplane could change its FORMATTING_PATTERN used to differentiate html containing formatting from html which can be replaced by plain text.

    Please tell me your preferences and reasons. As always I am particularly interested in seeing your concrete use cases which can not be realized with current behaviors.

    Regards,
    Dimitry

     
  • Milavon

    Milavon - 2021-12-05

    Thanks Dimitry
    I can now understand more about Freeplane behaviour and I'm sorry that I missed the compatibility issue.
    you are totally right about it. there is no need to break the compatiblity with Freemind or older Freeplane versions.
    Also, it is not a a significant issue at all. it was annoying for me at first but I used to find workarounds & scripts
    that enable me to handle large text very easly,
    and this is the beautiful part of freeplane:
    there are always workrounds and unlimited potentials opened for the users through scripting.

    So, there is no need to change anything, but as an answer for your suggestions:
    I think the second option: ("Freeplane could have a new explicit flag indicating "text/plain")
    looks more convenient, since it maintains the compatiblity, specially if this "text/plain" flag is implemented (when the user select the (Text) content type for any of the node parts: (core or notes or details)

    I'm not sure that I understood the third suggestion (Freeplane could change its FORMATTING_PATTERN ..)
    anyway, the second option is fine.

    Also, in my previous post: I didn't mean to suggest the idea of adding line breaks inside the node core text in the XML file, that would be very bad idea because it is far better to keep the node core text in a single line (inside the TEXT="" element),
    Currently, the node core text is handled very well & no problem about it

    But, now as it turned out: it is fine to keep them represented in HTML to maintain compatiblity.

    my point was:
    all about the performance issue when handling a plain text compared to any text editor,
    for example: when editing a large note through the (Note Dialog Editor)
    for example: if you opened (Open note in dialog) window & pasted a 5 MB of text in the note dialog editor window,
    Freeplane would freeze for sometime & the RAM size would increase to very high level, also the processor load.
    However, if you pasted the same amount of text to any text editor, it would be pasted easely & quickly
    (without too much RAM or processor load)
    So, the point was: in order for SimplyHTML to handle plain text effectively (in high performance),
    it should act simply like any text editor (without doing any extra operations on the text that would slow it & increase the load)
    and my understanding that this low performance with the large text came from the required conversion & parsing operations
    for the HTML text in order to display it as a plain text & vice versa

    However, for moe this performance issue is no longer a problem at all,
    because now when I want to paste a large text into a note,
    I avoid opening the note dialog editor to paste it. Instead, I use a script to paste the text from the outside into the note,
    and do the same thing to get it back

    Regards

     
  • Dimitry Polivaev

    Could you please explain why you need to paste so much text in note dialog?
    Without a concrete use case this discussion seems to be pointless.

    Regards

     
    • Milavon

      Milavon - 2021-12-07

      Yes, actually in my work, I collect a lot of log files & html web pages (saved as single .html files)
      for a long time, I was preferring to keep all those file contents inside freeplane map (to get everything in in one place / one file).
      meaning that I was using Freeplane as a storage container for any textual data.
      because, I was obsessed with the idea of adding all readable file formats content into freeplane notes as a plain text
      So, for example: an html file would copied as a plain text html code, and not as a rich text.

      most of these files would be not that large, but a few of them could be multiple megabytes

      however, it turned out that it is not a good idea at all to save files contents inside Freeplane, since it is not designed for that purpose even if it can do it.
      because that would dramatically increase the file size & RAM load.
      Now I think it is better to use Freeplane nodes as links to external files, allowing the files to be opened directly with their default relevant applications. instead of putting everything at freeplane map.

      So, it was not a concrete use case, rather than an obsession of having everything at one file.
      which now I realize it is not a good idea

      Thanks