#637 Use the small custom field only for bibliographic referencing

[Mark Grimshaw]

In v6.7.2, there are two types of custom field that can be added: a small VARCHAR(1020) text input field and a large mediumtext textarea field. Both are displayed when viewing a resource and both can be added to the bibliographic templates of a citation style.

Change this:
1. The small field is the only one to be used for citation styles.
2. The large one is the only one to be displayed when viewing a resource.

For fields displayed when viewing a resource there is no reason for two types. Equally, it is silly to use a large field to add to a bibliographic template. This proposed alteration clarifies matters.

NB
* The conversion process needs to transfer current 'S' custom fields to 'L'.
* When viewing a resource, ensure the small field is present when hovering over 'View all bibliographic details.'

If deciding to convert the 'S' to 'L' there might be problems if a 'S' field is already used for bibliographic template purposes. Even an 'L' field used for this purpose will present problems when the change occurs. The solution is to provide the opportunity to convert between 'S' and 'L' fields in the Admin interface.

[Stéphane Aulery]

Another problem must be taken into account. A 'L' field is always an HTML field. Firstly the modification changes the nature of the field. Secondly is it good not to be able to create a non-HTML custom field?

[Mark Grimshaw]

The 'L' field will still be used for what it was always used for. What the change really does is stop an HTML field being used for bibliography purposes.

Mark

[Stéphane Aulery]

In my opinion it is enough to create a new type of non-HTML custom field which cannot be used by the styles.

[Mark Grimshaw]

Well, in essence, that is what the 'S' field is for. I've been wondering for some time what the point was of having both 'S' and 'L' fields doing the same job.

Can you provide an instance where it is absolutely necessary to have a 'S' field that is non-HTML that is not used for the bibliography where the 'L' field would not be adequate?

[Stéphane Aulery]

Noting the number of a book in the library or its location is an example.

But what bothers me the most is the automatic conversion of existing S fields into L fields. You are assuming that none of these fields were used in a template.

[Mark Grimshaw]

I did think about that and have a) provided the option to convert 'L' (back) to 'S' and b) made a note in the CHANGELOG to that effect.

It's actually something that's been bothering me for a long time—having an 'S' field that was not in any way used differently to an 'L' field.

How about this? Tomorrow I create a 'B' (for bibliography) field and reinstate the 'S' field as it was before. There will be very few people who will have used the SVN to update their production server especially over the past couple of days.

Mark

[Stéphane Aulery]

It's actually something that's been bothering me for a long time—having an 'S' field that was not in any way used differently to an 'L' field.

Is it the presence of two types of fields or the presence of two database fields that bothers you?

[Mark Grimshaw]

1. It's the presence of two types of field serving the same purpose (display on the resource view) when just one type would do. The only advantage I see is that the 'S' field takes up less db space.
2. It's the possibility to use the 'L' field for the bibliography template. Thus the risk of images, tables, lists, etc. in the reference.

[Stéphane Aulery]

There is a third dimension. HTML/non-HTML. An HTML field is always more problematic: special treatment, length taking into account the code, security.

The mediumtext type consumes just 2 bytes more than the varchar type. Read this text to understand the real differences between varchar and (medium)text.

There are four dimensions to take into account: functionality, code complexity (+ security + maintainability), database implementation (optimizing access is more important than optimizing storage size because disks hard drives are less expensive than RAM and CPU).

Taking all of this into account, if the search is not important for these fields, we can simplify with a single mediumtext field in the database.

But the fact that the problem of changing the HTML/non-HTML type does not shock you makes me wonder about the correct definition of the functionality. One should first define the desired functionality precisely before implementing or re-implementing. Otherwise the functionality is distorted to fit the implementation.

(Reposted with links)

[Mark Grimshaw]

Fine with the memory then.

Over the years, I've come across wikindices using the custom fields for anything from single words or numbers to copious amounts of text including HTML formatting. So there is a need to have custom fields with HTML formatting for additional information when displaying a resource. Thus the 'L' type.

The other functionality of a custom field is to have extra fields available when formatting bibliographic entries—for example, columns or perhaps some library's catalogue number. As far as possible, these should not be HTML fields—so no formatting and certainly no images, tables, lists, etc. Such fields will not contain large amounts of text. Thus the 'S' type.

Note that my changes have not converted HTML to non-HTML. the 'S' type was always non-HTML using textinput instead of a tinyMCE-enabled textarea (the 'L' type).

Regards,

Mark

[Stéphane Aulery]

Note that my changes have not converted HTML to non-HTML. the 'S' type was always non-HTML using textinput instead of a tinyMCE-enabled textarea (the 'L' type).

In the upgrade code you change S fields to L fields. So this is a mistake?

[Mark Grimshaw]

No. In most cases that I have seen, the 'S' field is used for resource views so I made the decision to assume that that was likely to be the case. Only 'L' fields can now be displayed in the resource view thus I converted the 'S' fields to 'L' fields. In case this is not what a user wants—they are using the 'S' field for bibliography template purposes—I have provided the option to convert between 'L' and 'S' types. In other words, to rollback on my 'S' to 'L' conversion. I have noted this in the change log.

I do notice an error in my recent code should the user do this conversion—a change from 'L' to 'S' should use \HTML\stripHtml() on the appropriate resourcecustomLong fields, transfer those field values to resourcecustomShort, and set the resourcecustomLong fields to NULL. I will fix this later on.

Mark

[Stéphane Aulery]

If I take up the two problems you raise: sizes and display of complex data (tables, lists, etc.).

The first isn't really one. Knowing whether the field should be searched is even more so.

The second rather suggests that whatever the type of field, none should be HTML, but that there should be a personalized field such as notes, musings, quotes and others whose name is customizable and the content is HTML.

[Mark Grimshaw]

Both fields should be, and are, searchable.

Re your second point. The 'L' type is not specific to any one resource (like a musing, which, in any case, does what you suggest). It's a field available to all resources which can be populated or not. It can be edited from the resource view. It can also be edited from the resource edit page where the 'S' type can also be edited.

There is another advantage to using custom fields in the way they are currently used. Bibliography lists for importing, such as bibTeX, mght have extra fields that are not internally mapped in WIKINDX. An example might be an ID number. The import process gives the user the otption to map such unrecognised fields to other fields such as custom fields. This would not be possible using fields such as musing etc. which are for specific information regarding the resource and can have page numbers, keywords, and so on.

[Stéphane Aulery]

Only 'L' fields can now be displayed in the resource view thus I converted the 'S' fields to 'L' fields. In case this is not what a user wants—they are using the 'S' field for bibliography template purposes—I have provided the option to convert between 'L' and 'S' types. In other words, to rollback on my 'S' to 'L' conversion. I have noted this in the change log.

This is where I disagree. Users don't read the changelog first. If they encounter a problem then they read the changelog. In this case they will have started entering HTML into a field that they would like to be text.

In my opinion it can have a single storage field of type mediumtext (simpler for queries) and 3 types of custom field:
- S: display of one line of text (textinput)
- L: display of many lines of text (textearea)
- H: HTML display (textearea + TinyMCE)

Then you can hardcode the use of the S/L/H types depending on the target (bibliographic, resource view) or add booleans to signify this and make it customizable too (if a field is too long and displays poorly or is poorly formatted, this will be the user's choice).

[Mark Grimshaw]

I made the initial mistake years ago when I first implemented custom fields. There's also been feature creep when I later added the possibility to add the fields to bibliographic templates—they were there initially just to display extra information in resource view.

This is, I admit, a brute force change that runs a very small risk of confusing users, especially admins who don't, but should, read the CHANGELOG. I took the gamble and the new coding does what it should be doing. I've removed the option to edit the 'S' field from the resource view so it is less likely a user will start to edit it for that resource. And, remember, the 'S' field was never HTML so a user would not have been able to add HTML unless they did this by hand.

I think your proposal of the three types considerably complicates the coding and is overkill for what is required: HTML-enabled display for the resource view and a plain-text field that can be added to bibliographic templates. From your responses, I have the feeling I have not sufficiently explained how custom fields were initially coded and how they are intended to be used.

Yes, I should have left the issue here longer for you to comment on before implementing—apologies. But it had been at the back of my mind for a long time to make this change and I had a few hours available to accomplish it.

What we could do that would go some way to meeting your suggestion would be to merge the resourcecustomShort and resourcecustomLong columns into a single mediumtext resourcecustomText column.

Mark

Mark

[Stéphane Aulery]

I was also negligent in not reading this ticket sooner. Let's merge the database fields and keep the other elements as you did.

There is the same problem with the BibTeX field and probably with many other things. This is why I explained that we need to look at the functionality carefully before reviewing the implementation.

[Mark Grimshaw]

Agreed. And I'll be more patient in future! I'll make the fix today.

[Stéphane Aulery]

I believe we need to change our perspective and better define what you want for WIKINDX in terms of functionality, going from the general to the specific. Otherwise I raise technical problems which are not important.

If there are other things that are wrong create tickets.