I prepared a patch supporting a new symbology: playing cards. I explained all in a new doxygen page which I attach here. It explains the question I were asked. The patch is not yet fully tested, but I put it as a subject to discussion.
Can you also provide the full specification for the 53 encodings and some sample images?
> The barcode standard is not publicly documented.
This concerns me a little... Are you sure this isn't someone's private, protected (eg patented) technology?
> I got into touch with 2 different standards
Does this mean you contacted manufacturers and they gave you the specification? Did you explicitly obtain permission to publish an open source implementation of it? Just trying to cover our butts here...
> I decided to choose only one orientation (up) to avoid problems of mistaking 2 and 5 of spades.
A more flexible approach would be to report two linked symbols with the decode results for each orientation, giving the user full information to choose. fwiw- Pharmacode has the same assumption (controlled scanning environment) with the same ambiguity (different result based on scan direction) and this is the plan to handle it.
> That breaks the rule of zbar which usually reads barcodes in four possible orientations
I wouldn't call that a "rule of zbar", rather just an ambiguity in the symbology; looking only at an image of this barcode, it is impossible to tell which way is up and hence what the expected data should be.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>> The barcode standard is not publicly documented.
>This concerns me a little... Are you sure this isn't someone's private,
protected (eg patented) technology?
I'm not sure, I even have no idea. But I see nothing wrong in writing and publishing software capable of reading these simple codes, which do not carry any confidential information. Maybe my imagination and law knowledge is too short here. Maybe someone could do something wrong with this software later, but I hope we're not responsible for that. By googling for "reliable card dealing machines" you find the company that probably invented this type of barcoded playing cards. But they are copied by many manufacturers. I got mine legally in Poland, they originate from China. Please decide whether we can proceed further.
>> I got into touch with 2 different standards
>Does this mean you contacted manufacturers and they gave you the
specification? Did you explicitly obtain permission to publish an open
source implementation of it? Just trying to cover our butts here...
No. No. Take it literally :) I had them in hands while playing live bridge tournaments.
Here's the webpage selling our type cards: http://www.jannersten.com/html/cards.html. They provide pictures.
>A more flexible approach would be to report two linked symbols with the
decode results for each orientation, giving the user full information to
choose.
I don't see a place for the orienation: the presentation currently consists of 2 parts: symbology name and the symbol itself. Like PCARD:SA. Where would we put the orientation? Wouldn't it be confusing? Maybe giving the possibility of orientation configuration through command line would be better? Anyway I can adjust to your plan. My thought was too keep the output clear and force the user to present the card oriented properly.
>looking only at an image of this barcode, it is impossible to
tell which way is up and hence what the expected data should be.
Maybe pcard case is different as the symbol is always on the top of a card. We don't need to consider reading the symbol without a card.
By the way, feel free to correct my language errors in the project files any time.
I'll provide the pictures and the specification next time. Maybe then the legal state would be clear yet.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Browsing card sellers' pages I discovered they distinct 2 types of barcodes: DOS and WIN. The standard I prepared is WIN. I'm going to mention this in doc. What do you think about terminology, Spadix? Should we replace PCARD with something else?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> I see nothing wrong in writing and publishing software capable of reading these simple codes, which do not carry any confidential information.
It's nothing to do with confidentiality or security. If some entity invented and patented this technology then it is inappropriate for us to re-implement it. You find at least one example if you search "playing card barcode patent"
> Please decide whether we can proceed further.
It's your work, so I leave that to your discretion. If there is a patent, then my understanding is that the responsibility (in the US anyway) falls on the patent holder to address violations. Anyway, I'm not a patent lawyer; if you do not believe there is a problem, then by all means proceed.
> the presentation currently consists of 2 parts: symbology name and the symbol itself.
You mean the "symbology name and the decoded data" - we use "symbol" to refer to the graphical instance of a barcode and its attributes. The symbol object has more information than the symbology and data attributes printed by default from the applications. Use the --xml switch to see structured output with additional information, including the orientation. When you link the alternate representation as a component, it will not appear in the default output but will be available to clients of the library (the full structure should be available in the xml, but that has yet to be implemented).
> We don't need to consider reading the symbol without a card.
Where is the card in the image? How was the camera oriented wrt the card? We don't have that information, so we should provide the necessary detail for upper layers (which eventually do have that information) to apply.
> I discovered they distinct 2 types of barcodes: DOS and WIN. ... Should we replace PCARD with something else?
PCARD is fine. If it's unlikely that DOS (*DOS*?!) will be implemented then we should avoid making a special case for it now; we can always deal with it if it becomes an option.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>You find at least one example if you search "playing card
barcode patent"
Interesting hint Spadix. I found 2 patents for cards from J.. One for DOS (1994, now outdated) and one for WIN. Looks like the dealing machine is not patented, at least not by J. So it seems to be no danger from patents' side :)
I agree that xml could carry additional information about the symbols read. But please make your vision clear. How shall I change the treatment of found symbol, what to output in the default mode, where to discard the improperly orientated symbols, what switches to introduce. The current solution is fine for me, but I want it to be such for you too.
How should I post next patches now? To post subsequent patches to apply over the previous or to post just one full patch?
I'll supply the examples soon. I don't know what do you exactly want to include in the repository, so I'll provide a link to a set of images to choose from.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> xml could carry additional information about the symbols read
Not just xml, but the symbol object, of which xml is just one representation. You should think of ZBar first as a library with object APIs and only second as a suite of applications with some specific text output.
> How shall I change the treatment of found symbol
You should indicate the orientation, similar to other symbologies. If orientation is ambiguous for a specific case, then ideally you should arrange for multiple results to be available, one with each possible orientation, using the symbol component infrastructure. The disinterested zbarcam/zbarimg user will still see only the same "default" result (which assumes they understand the orientation constraints), while the Python/Perl/C++/etc programmer will have all the information they need to include the assumptions of their specific environment.
> what to output in the default mode
In this case you have a choice how to setup the component hierarchy, you might:
1. Use your selected "default" orientation as the parent symbol and add the alternate result as a component of the default.
2♠ (UP)
+-> 5♠ (DOWN)
2. Introduce a new parent symbol to encapsulate the ambiguity and add specific results as components
2♠/5♠ (UNKNOWN (or UP?))
+-> 2♠ (UP)
+-> 5♠ (DOWN)
Where the "default" output would be the top-level symbol and the components require extra effort to get at. fwiw- I think the second choice is cleaner...
> where to discard the improperly orientated symbols, what switches to introduce
Nothing should be discarded, and I'm not sure why new switches would be needed?
> post subsequent patches to apply over the previous or to post just one full patch?
It is up to you whether you would rather treat orientation handling as a separate patch. The patch supplied here should at least be tested, whether it includes orientation support or not. If you want to add/test orientation later as a separate feature, you can just open a new patch.
> I don't know what do you exactly want to include in the repository, so I'll provide a link to a set of images to choose from.
Test images are published in the test suite (the one run using "make regress"), which is separate from the source repository. You should select a reasonably sized set of images that provides good test coverage. Ideally you can also create the test index so I don't have to.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm beginning to understand. It will take some time to cover the orientation details. I will need a new test environment as I didn't deal with the object library yet and concentrated on text output instead. I hope I'll be ready before the 0.11 release. Please let me know if you plan to release it soon.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Spadix, could we commit pcard without the orientation support? I'm still not convinced to it, but if at least one innocent soul appears who would use this feature, I'll try to make it for him.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just wanted to let you know that I'm still looking through this patch. So far I can say that you did a great job with attention to detail! More feedback when I get a chance...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I finally made some time to look at this; my apologies again for the long latency.
I'll say up front that this is one of the best diffs I've ever received, esp regarding attention to detail and consistency. I only have a few comments that need to be addressed before we patch it in:
* You did a good job finding most of the places to touch, updating docs, etc. There are only a few missing; you can find most of them with, eg "grep -sirl code[- ]128". Coarsly:
- mention in README, README.windows
- python README
- iphone README, docs
- perl support (README, Symbol.pod, ZBar.pm, ZBar.xs)
- java support (just enum value in Symbol.java)
I don't think these have to be in this patch, but they'll need to be addressed before we can release.
* New files (at least) need your name for the copyright (and dated only in 2011)
* The symbology hash needs to be updated with the new enum value. I will attach the regenerated table for you.
* The runtime default for this symbology should be disabled.
* The decode lookup array is too space inefficient - this table statically occupies >2K, of which it only uses a handful of values from a limited range. A better approach would decode the element combinations (instead of the module combinations), since that space is much more compact. Another alternative might be a compressed hash, but finding that can be a challenge.
* Decode table entries should also be compressed to single bytes.
* Why is macro_fill_array_codes a macro? (should be static inline)
* What does pcard->done do? (looks like a dead store)
* The longest symbol fits entirely within the decode window, there is no need to track state across cycles (ie, remove idx_start).
* The ratio magic numbers can be constants, they would not be exposed outside this module.
* Avoid percentages - no need to waste bits for constant fixed point ratios. Now that you've found your magic number, round off to a ratio with fewer bits.
* Comparing bars to spaces assumes a perfect exposure, which probably explains the threshold tweaking and your related performance comment. You can avoid this entirely by using bar+space pairs (like-edges) or looking at bars and spaces separately. The latter also leads into the element-wise combination decode suggested above...
Looking forward to the updates so we can get this patch in!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'll be out for holiday from 13 to 27 of August. I guess you want to close it before I leave. That's why I post a patch before I fully test it and prepare for final deployment. I want to present the essential changes I made.
* patching other files - done
* my name - done
>* The symbology hash needs to be updated with the new enum value. I will
>attach the regenerated table for you.
do it, I inserted 53
>* The runtime default for this symbology should be disabled.
done
>* The decode lookup array is too space inefficient - this table statically
Reduced its size to 53*4. It compromises speed of checking whether bit_code fits any symbol this 53 times.
>* Decode table entries should also be compressed to single bytes.
We could save 53 bytes minus the bytes for new code to make for it. I don't think its worth it, taking into consideration amount of work needed and code complication.
>* Why is macro_fill_array_codes a macro? (should be static inline)
This macro is to be deleted after I make tests.
>* What does pcard->done do? (looks like a dead store)
done. And it's so dead that I even can't remember what was it for :)
>* The longest symbol fits entirely within the decode window, there is no
>need to track state across cycles (ie, remove idx_start).
idx_start serves as a marker. Negative value denotes nothing found yet. Ok?
>* The ratio magic numbers can be constants, they would not be exposed
>outside this module.
If one wants to experiment he could change it on the fly. Maybe we leave it that way?
>* Avoid percentages - no need to waste bits for constant fixed point
>ratios. Now that you've found your magic number, round off to a ratio with
>fewer bits.
I'm not sure yet whether the value is best. Let's leave it this way for some time. It would make further experiments easier. The reason is we have no specification for the standard, hence the probable need of adjustments.
>* Comparing bars to spaces assumes a perfect exposure, which probably
>explains the threshold tweaking and your related performance comment. You
>can avoid this entirely by using bar+space pairs (like-edges) or looking at
>bars and spaces separately. The latter also leads into the element-wise
>combination decode suggested above...
My goal was to limit false symbol detections. If I don't check for equalness of bars and spaces I will have more false symbols, won't I?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think that's ready to patch in, provided you accepted my previous answers.
Maybe the fill_array macro may remain in pcard_defs.h? Now we'll have 2 alternative ways of testing decoder - the old table and a new one. The old table is not contained in the library, because it's a macro.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Excellent, thanks for the detailed explanation!
Can you also provide the full specification for the 53 encodings and some sample images?
> The barcode standard is not publicly documented.
This concerns me a little... Are you sure this isn't someone's private, protected (eg patented) technology?
> I got into touch with 2 different standards
Does this mean you contacted manufacturers and they gave you the specification? Did you explicitly obtain permission to publish an open source implementation of it? Just trying to cover our butts here...
> I decided to choose only one orientation (up) to avoid problems of mistaking 2 and 5 of spades.
A more flexible approach would be to report two linked symbols with the decode results for each orientation, giving the user full information to choose. fwiw- Pharmacode has the same assumption (controlled scanning environment) with the same ambiguity (different result based on scan direction) and this is the plan to handle it.
> That breaks the rule of zbar which usually reads barcodes in four possible orientations
I wouldn't call that a "rule of zbar", rather just an ambiguity in the symbology; looking only at an image of this barcode, it is impossible to tell which way is up and hence what the expected data should be.
2011-05-05
Spadix
>> The barcode standard is not publicly documented.
>This concerns me a little... Are you sure this isn't someone's private,
protected (eg patented) technology?
I'm not sure, I even have no idea. But I see nothing wrong in writing and publishing software capable of reading these simple codes, which do not carry any confidential information. Maybe my imagination and law knowledge is too short here. Maybe someone could do something wrong with this software later, but I hope we're not responsible for that. By googling for "reliable card dealing machines" you find the company that probably invented this type of barcoded playing cards. But they are copied by many manufacturers. I got mine legally in Poland, they originate from China. Please decide whether we can proceed further.
>> I got into touch with 2 different standards
>Does this mean you contacted manufacturers and they gave you the
specification? Did you explicitly obtain permission to publish an open
source implementation of it? Just trying to cover our butts here...
No. No. Take it literally :) I had them in hands while playing live bridge tournaments.
Here's the webpage selling our type cards: http://www.jannersten.com/html/cards.html. They provide pictures.
>A more flexible approach would be to report two linked symbols with the
decode results for each orientation, giving the user full information to
choose.
I don't see a place for the orienation: the presentation currently consists of 2 parts: symbology name and the symbol itself. Like PCARD:SA. Where would we put the orientation? Wouldn't it be confusing? Maybe giving the possibility of orientation configuration through command line would be better? Anyway I can adjust to your plan. My thought was too keep the output clear and force the user to present the card oriented properly.
>looking only at an image of this barcode, it is impossible to
tell which way is up and hence what the expected data should be.
Maybe pcard case is different as the symbol is always on the top of a card. We don't need to consider reading the symbol without a card.
By the way, feel free to correct my language errors in the project files any time.
I'll provide the pictures and the specification next time. Maybe then the legal state would be clear yet.
Browsing card sellers' pages I discovered they distinct 2 types of barcodes: DOS and WIN. The standard I prepared is WIN. I'm going to mention this in doc. What do you think about terminology, Spadix? Should we replace PCARD with something else?
> I see nothing wrong in writing and publishing software capable of reading these simple codes, which do not carry any confidential information.
It's nothing to do with confidentiality or security. If some entity invented and patented this technology then it is inappropriate for us to re-implement it. You find at least one example if you search "playing card barcode patent"
> Please decide whether we can proceed further.
It's your work, so I leave that to your discretion. If there is a patent, then my understanding is that the responsibility (in the US anyway) falls on the patent holder to address violations. Anyway, I'm not a patent lawyer; if you do not believe there is a problem, then by all means proceed.
> the presentation currently consists of 2 parts: symbology name and the symbol itself.
You mean the "symbology name and the decoded data" - we use "symbol" to refer to the graphical instance of a barcode and its attributes. The symbol object has more information than the symbology and data attributes printed by default from the applications. Use the --xml switch to see structured output with additional information, including the orientation. When you link the alternate representation as a component, it will not appear in the default output but will be available to clients of the library (the full structure should be available in the xml, but that has yet to be implemented).
> We don't need to consider reading the symbol without a card.
Where is the card in the image? How was the camera oriented wrt the card? We don't have that information, so we should provide the necessary detail for upper layers (which eventually do have that information) to apply.
> I discovered they distinct 2 types of barcodes: DOS and WIN. ... Should we replace PCARD with something else?
PCARD is fine. If it's unlikely that DOS (*DOS*?!) will be implemented then we should avoid making a special case for it now; we can always deal with it if it becomes an option.
>You find at least one example if you search "playing card
barcode patent"
Interesting hint Spadix. I found 2 patents for cards from J.. One for DOS (1994, now outdated) and one for WIN. Looks like the dealing machine is not patented, at least not by J. So it seems to be no danger from patents' side :)
I agree that xml could carry additional information about the symbols read. But please make your vision clear. How shall I change the treatment of found symbol, what to output in the default mode, where to discard the improperly orientated symbols, what switches to introduce. The current solution is fine for me, but I want it to be such for you too.
How should I post next patches now? To post subsequent patches to apply over the previous or to post just one full patch?
I'll supply the examples soon. I don't know what do you exactly want to include in the repository, so I'll provide a link to a set of images to choose from.
> xml could carry additional information about the symbols read
Not just xml, but the symbol object, of which xml is just one representation. You should think of ZBar first as a library with object APIs and only second as a suite of applications with some specific text output.
> How shall I change the treatment of found symbol
You should indicate the orientation, similar to other symbologies. If orientation is ambiguous for a specific case, then ideally you should arrange for multiple results to be available, one with each possible orientation, using the symbol component infrastructure. The disinterested zbarcam/zbarimg user will still see only the same "default" result (which assumes they understand the orientation constraints), while the Python/Perl/C++/etc programmer will have all the information they need to include the assumptions of their specific environment.
> what to output in the default mode
In this case you have a choice how to setup the component hierarchy, you might:
1. Use your selected "default" orientation as the parent symbol and add the alternate result as a component of the default.
2♠ (UP)
+-> 5♠ (DOWN)
2. Introduce a new parent symbol to encapsulate the ambiguity and add specific results as components
2♠/5♠ (UNKNOWN (or UP?))
+-> 2♠ (UP)
+-> 5♠ (DOWN)
Where the "default" output would be the top-level symbol and the components require extra effort to get at. fwiw- I think the second choice is cleaner...
> where to discard the improperly orientated symbols, what switches to introduce
Nothing should be discarded, and I'm not sure why new switches would be needed?
> post subsequent patches to apply over the previous or to post just one full patch?
It is up to you whether you would rather treat orientation handling as a separate patch. The patch supplied here should at least be tested, whether it includes orientation support or not. If you want to add/test orientation later as a separate feature, you can just open a new patch.
> I don't know what do you exactly want to include in the repository, so I'll provide a link to a set of images to choose from.
Test images are published in the test suite (the one run using "make regress"), which is separate from the source repository. You should select a reasonably sized set of images that provides good test coverage. Ideally you can also create the test index so I don't have to.
picture of 9 cards
I'm beginning to understand. It will take some time to cover the orientation details. I will need a new test environment as I didn't deal with the object library yet and concentrated on text output instead. I hope I'll be ready before the 0.11 release. Please let me know if you plan to release it soon.
Spadix, could we commit pcard without the orientation support? I'm still not convinced to it, but if at least one innocent soul appears who would use this feature, I'll try to make it for him.
> I will need a new test environment
Ideally we could use/enhance the existing image regression and decoder testing infrastructure...
> the 0.11 release. Please let me know if you plan to release it soon.
No hurry - the next release is still 2-3 months away.
> could we commit pcard without the orientation support?
Sure, makes sense to me. If you're happy w/the diffs, I'll start going over them.
doxygen page v. 1.0.13
main patch 1.0.13 - tested
pcard test_decode patch 1.0.5
pcard test suite
I think this is it. I'm ready for remarks.
Just wanted to let you know that I'm still looking through this patch. So far I can say that you did a great job with attention to detail! More feedback when I get a chance...
I finally made some time to look at this; my apologies again for the long latency.
I'll say up front that this is one of the best diffs I've ever received, esp regarding attention to detail and consistency. I only have a few comments that need to be addressed before we patch it in:
* You did a good job finding most of the places to touch, updating docs, etc. There are only a few missing; you can find most of them with, eg "grep -sirl code[- ]128". Coarsly:
- mention in README, README.windows
- python README
- iphone README, docs
- perl support (README, Symbol.pod, ZBar.pm, ZBar.xs)
- java support (just enum value in Symbol.java)
I don't think these have to be in this patch, but they'll need to be addressed before we can release.
* New files (at least) need your name for the copyright (and dated only in 2011)
* The symbology hash needs to be updated with the new enum value. I will attach the regenerated table for you.
* The runtime default for this symbology should be disabled.
* The decode lookup array is too space inefficient - this table statically occupies >2K, of which it only uses a handful of values from a limited range. A better approach would decode the element combinations (instead of the module combinations), since that space is much more compact. Another alternative might be a compressed hash, but finding that can be a challenge.
* Decode table entries should also be compressed to single bytes.
* Why is macro_fill_array_codes a macro? (should be static inline)
* What does pcard->done do? (looks like a dead store)
* The longest symbol fits entirely within the decode window, there is no need to track state across cycles (ie, remove idx_start).
* The ratio magic numbers can be constants, they would not be exposed outside this module.
* Avoid percentages - no need to waste bits for constant fixed point ratios. Now that you've found your magic number, round off to a ratio with fewer bits.
* Comparing bars to spaces assumes a perfect exposure, which probably explains the threshold tweaking and your related performance comment. You can avoid this entirely by using bar+space pairs (like-edges) or looking at bars and spaces separately. The latter also leads into the element-wise combination decode suggested above...
Looking forward to the updates so we can get this patch in!
Hi Spadix,
I'll be out for holiday from 13 to 27 of August. I guess you want to close it before I leave. That's why I post a patch before I fully test it and prepare for final deployment. I want to present the essential changes I made.
* patching other files - done
* my name - done
>* The symbology hash needs to be updated with the new enum value. I will
>attach the regenerated table for you.
do it, I inserted 53
>* The runtime default for this symbology should be disabled.
done
>* The decode lookup array is too space inefficient - this table statically
Reduced its size to 53*4. It compromises speed of checking whether bit_code fits any symbol this 53 times.
>* Decode table entries should also be compressed to single bytes.
We could save 53 bytes minus the bytes for new code to make for it. I don't think its worth it, taking into consideration amount of work needed and code complication.
>* Why is macro_fill_array_codes a macro? (should be static inline)
This macro is to be deleted after I make tests.
>* What does pcard->done do? (looks like a dead store)
done. And it's so dead that I even can't remember what was it for :)
>* The longest symbol fits entirely within the decode window, there is no
>need to track state across cycles (ie, remove idx_start).
idx_start serves as a marker. Negative value denotes nothing found yet. Ok?
>* The ratio magic numbers can be constants, they would not be exposed
>outside this module.
If one wants to experiment he could change it on the fly. Maybe we leave it that way?
>* Avoid percentages - no need to waste bits for constant fixed point
>ratios. Now that you've found your magic number, round off to a ratio with
>fewer bits.
I'm not sure yet whether the value is best. Let's leave it this way for some time. It would make further experiments easier. The reason is we have no specification for the standard, hence the probable need of adjustments.
>* Comparing bars to spaces assumes a perfect exposure, which probably
>explains the threshold tweaking and your related performance comment. You
>can avoid this entirely by using bar+space pairs (like-edges) or looking at
>bars and spaces separately. The latter also leads into the element-wise
>combination decode suggested above...
My goal was to limit false symbol detections. If I don't check for equalness of bars and spaces I will have more false symbols, won't I?
>It compromises speed of checking whether bit_code fits any symbol this 53 times.
I think I'll presort symbol_defs and do a binary search on it. Relatively simple and optimal.
complete
I think that's ready to patch in, provided you accepted my previous answers.
Maybe the fill_array macro may remain in pcard_defs.h? Now we'll have 2 alternative ways of testing decoder - the old table and a new one. The old table is not contained in the library, because it's a macro.