#5 somebody set us up the bomb

[Anonymous]

I'm working on a migration script (transformation between different database structures) and I've just received the following error message:

PHP Fatal error: somebody set us up the bomb in /.../markdownify.php on line 913

Does anybody have an idea what may cause this error message?

[Milian Wolff]

Logged In: YES
user_id=1594854
Originator: NO

I deduce from your line number that you are not using current SVN. Could you try it with that again?

If your problem persists - can you give me a (hopefully stripped down) test case of your HTML input?

[Nobody/Anonymous]

Logged In: NO

Hi again,

I've updated the source code and have now recived some new warnings and errors that I hope some may help me with:

1. PHP Warning: call_user_func(Markdownify_Extra::flushStacked_ul): First argument is expected to be a valid callback in /usr/local/www/.../markdownify/markdownify.php on line 392

And this is the data:

<ul>
<li> Help file started
<li> Lots of bugs fixed
<li> Extended tools and extensions editor
<li> New drawing tools added (flood fill)
<li> Extension (Tools) templates added
<li> Clipboard cut and paste added
<li> Started work on a GCC exporter example
<li> UI advanced further
<li> New tools added
<li> Previous run state storage cleaned up a bit more
</ul>

And this is the output:

* Help file started
* Lots of bugs fixed
* Extended tools and extensions editor
* New drawing tools added (flood fill)
* Extension (Tools) templates added
* Clipboard cut and paste added
* Started work on a GCC exporter example
* UI advanced further
* New tools added
* Previous run state storage cleaned up a bit more </ul>

2. PHP Warning: buffer was not flushed, this is a bug. please report! in /usr/local/www/.../markdownify/markdownify.php on line 330

Data input:

<table align=center width=80% cellpadding=2><tr><td><hr><font size=1 color=red><b>NB!</b></font> <font size=1 color=#666666><b>Title
Until </font><font size=1 color=black><b>friday march 14.</b></font><font size=1 color=#666666><b>Long text. Long text</b></font><hr></td></tr></table>

Data output:

<table align=center width=80% cellpadding=2><tr><td><hr><font size=1 color=red><b>NB!</b></font> <font size=1 color=#666666><b>Title
Until </font><font size=1 color=black><b>friday march 14.</b></font><font size=1 color=#666666><b>Long text. Long text</b></font><hr></td></tr></table>

Do you know what may cause these problems?

[Milian Wolff]

Logged In: YES
user_id=1594854
Originator: NO

Oh... that does not look good at all.

Regarding the first PHP warning:
I will have to make my HTML parser much more sophisticated to parse HTML with non-closed tags... I'm not sure I want to do this. Could you use XHTML syntax which explicitly closes tags (`<li>...</li>`) as a work around? This is still valid HTML (and in my eyes even much cleaner HTML) and makes parsing much easier.

To the second error: It shows multiple shortcomings of Markdownify as is
- incorrect markup messes up the HTML parser, you are missing a `</b>` in the snippet
- Markdown (Extra) has no support for some of the features in your input, the font tags, tag attributes etc. will all be dropped
- Markdown Extra only supports a very specific type of tables, i.e. one row of headers and data below, you have a single cell with no header => won't work!

The problem is - how does one detect these kinds of tables? What to do with them? Converting something like Markdown to HTML is easy because the latter can do more than the former. Doing it the other way around many undefined cases crop up.

I personally think Markdownify is not an optimal solution for a database migration script with HTML like this. I see it more like a complement to HTML generated by Markdown. Or "simple" HTML like used for CMS contents, which relies on CSS for styling. What you are posting there is pretty much the worst-case...

Since you use it in a migration script, speed is not a limit. What about using http://php.net/tidy (or similar) before pushing the HTML to Markdownify?