Universal JTAG library, server and tools

On Sat, Mar 21, 2009 at 06:00:33PM -0300, Jesus Sanchez-Palencia wrote:

> On your documentation I saw that is a future plan for UrJTAG to have its
> code base as a library so other applications can use it. But on your home I
> saw this "(...) got rid of many global variables, which is a major
> improvement for everyone who wants to use UrJTAG as a library". So, I was
> wondering, is it possible right now to use urjtag as a library or not ?!

Converting UrJTAG into a library is quite straightforward. I've done
this already and it works quite well for me. I'm planning to contribute
the code to UrJTAG project, but there's still a lot to change and
I'm swamped with other work right now (http://labserver.uv.es/ioboard/).

You may get my code from svn:
$ svn co http://labserver.uv.es/svn_FPGA/trunk/source/urjtag urjtag-lib

AFAIR it is based on revision 1407. The main changes are:
- changed autotools sources (configure.ac, Makefile.am) to
  use libtool
- moved some code from jtag.c to liburjtag.c
- moved code unused in the library from from cmd/cmd.c to jtag.c
- command-line app now uses the library
- added XS code to build a perl module

What needs to be done (before adding it to official UrJTAG tree):
- sanitise namespace (hide unused functions, prefix all the used
  ones with jtag_)
- cleanup and install public headers
- perl: allow to call any function usable within perl

I'll be working on that, but for now I've got to finish the PCB, which
may take few more weeks.


Good luck with your project !


-- 
 ____  Sparky{PI] -- Przemyslaw _  ___  _  _  ........... LANG...Pl..Ca..Es..En
/____) ___  ___  _ _ || Iskra  |  | _ \| |  | : http://www........ppcrcd.pld-linux.org
\____\| -_)'___| ||^'||//\\// <   |  _/| |  | : JID......sparky<at>jabberes.org
(____/||   (_-_|_||  ||\\ ||   |_ |_|  |_| _| : Mail....sparky<at>pld-linux.org

Great! I'll take a look at it, for sure!

What about the Olimex LPC-E2124 (http://www.olimex.com/dev/lpc-e2124.html)
support ? Does anyone know anything about this ?


Thanks again!
Jesus

2009/3/22 Przemyslaw Iskra <sparky@...>

> On Sat, Mar 21, 2009 at 06:00:33PM -0300, Jesus Sanchez-Palencia wrote:
>
> > On your documentation I saw that is a future plan for UrJTAG to have its
> > code base as a library so other applications can use it. But on your home
> I
> > saw this "(...) got rid of many global variables, which is a major
> > improvement for everyone who wants to use UrJTAG as a library". So, I was
> > wondering, is it possible right now to use urjtag as a library or not ?!
>
> Converting UrJTAG into a library is quite straightforward. I've done
> this already and it works quite well for me. I'm planning to contribute
> the code to UrJTAG project, but there's still a lot to change and
> I'm swamped with other work right now (http://labserver.uv.es/ioboard/).
>
> You may get my code from svn:
> $ svn co http://labserver.uv.es/svn_FPGA/trunk/source/urjtag urjtag-lib
>
> AFAIR it is based on revision 1407. The main changes are:
> - changed autotools sources (configure.ac, Makefile.am) to
>  use libtool
> - moved some code from jtag.c to liburjtag.c
> - moved code unused in the library from from cmd/cmd.c to jtag.c
> - command-line app now uses the library
> - added XS code to build a perl module
>
> What needs to be done (before adding it to official UrJTAG tree):
> - sanitise namespace (hide unused functions, prefix all the used
>  ones with jtag_)
> - cleanup and install public headers
> - perl: allow to call any function usable within perl
>
> I'll be working on that, but for now I've got to finish the PCB, which
> may take few more weeks.
>
>
> Good luck with your project !
>
>
> --
>  ____  Sparky{PI] -- Przemyslaw _  ___  _  _  ...........
> LANG...Pl..Ca..Es..En
> /____) ___  ___  _ _ || Iskra  |  | _ \| |  | : http://www........
> ppcrcd.pld-linux.org
> \____\| -_)'___| ||^'||//\\// <   |  _/| |  | : JID......sparky<at>
> jabberes.org
> (____/||   (_-_|_||  ||\\ ||   |_ |_|  |_| _| : Mail....sparky<at>
> pld-linux.org
>

Przemyslaw Iskra wrote:
> On Sat, Mar 21, 2009 at 06:00:33PM -0300, Jesus Sanchez-Palencia wrote:
> 
>> On your documentation I saw that is a future plan for UrJTAG to have its
>> code base as a library so other applications can use it. But on your home I
>> saw this "(...) got rid of many global variables, which is a major
>> improvement for everyone who wants to use UrJTAG as a library". So, I was
>> wondering, is it possible right now to use urjtag as a library or not ?!
> 
> Converting UrJTAG into a library is quite straightforward. I've done
> this already and it works quite well for me. I'm planning to contribute
> the code to UrJTAG project, but there's still a lot to change and
> I'm swamped with other work right now (http://labserver.uv.es/ioboard/).
> 
> You may get my code from svn:
> $ svn co http://labserver.uv.es/svn_FPGA/trunk/source/urjtag urjtag-lib
> 
> AFAIR it is based on revision 1407. The main changes are:
> - changed autotools sources (configure.ac, Makefile.am) to
>   use libtool
> - moved some code from jtag.c to liburjtag.c
> - moved code unused in the library from from cmd/cmd.c to jtag.c
> - command-line app now uses the library
> - added XS code to build a perl module
> 
> What needs to be done (before adding it to official UrJTAG tree):
> - sanitise namespace (hide unused functions, prefix all the used
>   ones with jtag_)
> - cleanup and install public headers
> - perl: allow to call any function usable within perl
> 
> I'll be working on that, but for now I've got to finish the PCB, which
> may take few more weeks.

Our project (http://www.rfidguardian.org) will also want to use UrJtag 
as a library -- it's a year ago that I worked on it previously, but now 
we are going to have a new version of the hardware and in the upcoming 
months, we will need this functionality.

Our board will use the microcontroller (a BlackFin BF547) to program an 
FPGA at boot time; the FPGA JTAG pins will be connected to GPIO pins of 
the BlackFin. Last year, I wrote a prototype GPIO 'cable' for our 
previous hardware version (with an XScale), it basically ran UrJtag on 
our microcontroller as a stand-alone program. But in the upcoming 
version, we want to integrate the UrJtag functionality into our application.

So, our plans:
- convert UrJtag to library state, exactly like Przemyslaw did/plans to do;
- do it during the next 3 or 4 months; it will have to share priority 
with a number of other tasks within the project;
- contribute our work, including the GPIO 'cable' for all that's worth

Is the path as laid out by Przemyslaw considered a good idea by the 
UrJtag community? I am aware that there are a lot of different issues to 
be solved to achieve the final library state, but this could arguably be 
a useful intermediate state, e.g. for people who are content w/ GPL as a 
licence, like we are.

So the question is: if Przemyslaw, Jesus, and/or me complete Premyslaw's 
stuff (and upgrade to bleeding edge trunk), will the result be pulled 
into UrJtag? Our project wants to contribute anything we do to the 
community; preferably by pushing it to UrJtag, but if this approach 
turns out to be unacceptable, we will publish it on our own svn server. 
The snag is of course that the changes will affect the APIs, and 
possibly a lot of places in the code.

Rutger

Well, seems that this is feature needed by lots of people out there!

Rutger's questions are mine also. In summary, is Przemyslaw's work
considered a fork or a not-yet-merged set of patches ?!

Rutger, If I decide to go on with this approach, adding the support I need
to UrJTAG and helping with the liburjtag, I'll need to do this all on April.
I must finish coding all my work until first week of June, since I will
still have to write my Diploma Thesis and present it at the university in
order to graduate and I have to accomplish this until June 20th.

We all could work on the same repository to avoid more forks of the same
project, but I still have to figure it out how to add ARM7TDMI support
before deciding how I'll proceed.

Jesus

2009/3/22 Rutger Hofman <rutger@...>

> Przemyslaw Iskra wrote:
>
>> On Sat, Mar 21, 2009 at 06:00:33PM -0300, Jesus Sanchez-Palencia wrote:
>>
>>  On your documentation I saw that is a future plan for UrJTAG to have its
>>> code base as a library so other applications can use it. But on your home
>>> I
>>> saw this "(...) got rid of many global variables, which is a major
>>> improvement for everyone who wants to use UrJTAG as a library". So, I was
>>> wondering, is it possible right now to use urjtag as a library or not ?!
>>>
>>
>> Converting UrJTAG into a library is quite straightforward. I've done
>> this already and it works quite well for me. I'm planning to contribute
>> the code to UrJTAG project, but there's still a lot to change and
>> I'm swamped with other work right now (http://labserver.uv.es/ioboard/).
>>
>> You may get my code from svn:
>> $ svn co http://labserver.uv.es/svn_FPGA/trunk/source/urjtag urjtag-lib
>>
>> AFAIR it is based on revision 1407. The main changes are:
>> - changed autotools sources (configure.ac, Makefile.am) to
>>  use libtool
>> - moved some code from jtag.c to liburjtag.c
>> - moved code unused in the library from from cmd/cmd.c to jtag.c
>> - command-line app now uses the library
>> - added XS code to build a perl module
>>
>> What needs to be done (before adding it to official UrJTAG tree):
>> - sanitise namespace (hide unused functions, prefix all the used
>>  ones with jtag_)
>> - cleanup and install public headers
>> - perl: allow to call any function usable within perl
>>
>> I'll be working on that, but for now I've got to finish the PCB, which
>> may take few more weeks.
>>
>
> Our project (http://www.rfidguardian.org) will also want to use UrJtag as
> a library -- it's a year ago that I worked on it previously, but now we are
> going to have a new version of the hardware and in the upcoming months, we
> will need this functionality.
>
> Our board will use the microcontroller (a BlackFin BF547) to program an
> FPGA at boot time; the FPGA JTAG pins will be connected to GPIO pins of the
> BlackFin. Last year, I wrote a prototype GPIO 'cable' for our previous
> hardware version (with an XScale), it basically ran UrJtag on our
> microcontroller as a stand-alone program. But in the upcoming version, we
> want to integrate the UrJtag functionality into our application.
>
> So, our plans:
> - convert UrJtag to library state, exactly like Przemyslaw did/plans to do;
> - do it during the next 3 or 4 months; it will have to share priority with
> a number of other tasks within the project;
> - contribute our work, including the GPIO 'cable' for all that's worth
>
> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag
> community? I am aware that there are a lot of different issues to be solved
> to achieve the final library state, but this could arguably be a useful
> intermediate state, e.g. for people who are content w/ GPL as a licence,
> like we are.
>
> So the question is: if Przemyslaw, Jesus, and/or me complete Premyslaw's
> stuff (and upgrade to bleeding edge trunk), will the result be pulled into
> UrJtag? Our project wants to contribute anything we do to the community;
> preferably by pushing it to UrJtag, but if this approach turns out to be
> unacceptable, we will publish it on our own svn server. The snag is of
> course that the changes will affect the APIs, and possibly a lot of places
> in the code.
>
> Rutger
>

> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag community? I am aware that there are a lot
> of different issues to be solved to achieve the final library state, but this could arguably be a useful intermediate

Hi,

I'm sorry I haven't commented much in the past, I'm very busy at work
and now will be on holiday until mid April, but on this important topic
I'd like to tell at least that I personally do favor the approach
described by Przemyslaw Iskra. IMHO it is the right way to go.

I once planned to design the library API begining with the design of an
efficient communication protocol, but as we know, that isn't done now
even after almost two years. So, the logical alternative is to base the
API design on what's already there in the code.

There is a minor issue with libtool - with libtool, the build process
can become very time consuming in Cygwin/MinGW environment. Switching to
CMake would help a lot in this regard - but that's no priority and
probably just a current personal preference.

In the long term, the interface to Perl and other languages could be
implemented using SWIG. While it certainly isn't perfect in every
regard, it can be of great help.

Kolja

Hi,

> same project, but I still have to figure it out how to add ARM7TDMI
> support before deciding how I'll proceed.

I have no experience with that particular chip; the LPCxxx from NXP are
often different with regard to their JTAG interface capabilities. Some
allow boundary scan, some do so with special pin configuration only,
many don't. We have no code in UrJTAG to interface with the ARM debug
core yet (as for MIPS, or as OpenOCD has for ARM), to inject and run
code, set breakpoints etc... but that might be necessary to access the
internal flash in the LPC.

Kolja

>> same project, but I still have to figure it out how to add ARM7TDMI
>> support before deciding how I'll proceed.
>
> I have no experience with that particular chip; the LPCxxx from NXP are
> often different with regard to their JTAG interface capabilities. Some
> allow boundary scan, some do so with special pin configuration only,
> many don't. We have no code in UrJTAG to interface with the ARM debug
> core yet (as for MIPS, or as OpenOCD has for ARM), to inject and run
> code, set breakpoints etc... but that might be necessary to access the
> internal flash in the LPC.

Jesus,

The short answer that Kolja is getting at, is that what you are trying
to attempt is probably a better fit for OpenOCD, although I have to
say, my experience with UrJtag has been rather less frustrating.
There is certainly room for some enterprising individual to combine
the strengths of each in some logical way (read: libraries), and it
has been discussed on this list...the primary difference between the
two camps (which has been perhaps somewhat blurred at this point) is
that the UrJtag group is primarily FPGA (programmable logic) users,
and the OpenOCD camp is mainly made up of processor users, which
affects the supported target devices and jtag dongles of each.  Also,
OpenOCD is more of a client/server model (to support connections to
GDB), whereas UrJtag is more of a standalone application.  OpenOCD is
has two distinct personalities, programming flash on the processor,
much like UrJtag for FPGA work, and (2), debugging a running program
on the target.  IMHO, the two aren't strictly compatible from the user
standpoint, although it is usable at this point...

I'm adding this somewhat off-topic reply to the UrJtag list; having
been a user of both, perhaps this 'experiential' knowledge will help
someone else in the future...if they read the archives...

Best,
Steve

>> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag community? I am aware that there are a lot
>> of different issues to be solved to achieve the final library state, but this could arguably be a useful intermediate
> 
> I'd like to tell at least that I personally do favor the approach
> described by Przemyslaw Iskra. IMHO it is the right way to go.
> 
> I once planned to design the library API begining with the design of an
> efficient communication protocol, but as we know, that isn't done now
> even after almost two years. So, the logical alternative is to base the
> API design on what's already there in the code.

I'd second that. Turning the jtag shell commands into API functions
would keep the implications low since there are not many fundamental
changes involved. It will be just the first of a series of steps
towards a well-featured UrJTAG library, however.

My basic view on this topic is to first do the transition to liburjtag.a
as shown in Przemyslaw's code. This includes redefining the command
shell interface to have commands separated from API functionality. At
the moment src/cmd/* implements shell commands AND related
functionality, this needs to change then.
Namespace clean-up and file renaming should happen here as well.

As a result, liburjtag would expose library functions that are
equivalent to the currently existing commands. I'd call this the
legacy/cmd API.

> In the long term, the interface to Perl and other languages could be
> implemented using SWIG. While it certainly isn't perfect in every
> regard, it can be of great help.

I agree here as well. I don't have any experience with SWIG but if it
can keep its promises, we'd quickly come up with bindings for a number
of languages.
One topic to consider in this context is the existing command line
interface of UrJTAG/jtag. While the shell itself might lose importance
over time, there's still the parts definition database. The definition
files use the command syntax and liburjtag would require to support them
as well, with or without the jtag shell.


Concerning the timeline, my idea is to release 0.10 soon. There's too
much accumulated in SVN already and I point support requests to SVN
check-outs too often in the meantime. Work on liburjtag won't make
things less complicated.
>From a personal perspective, I could support efforts for liburjtag best
in the second half of April.

Cheers
Arnim

Arnim Läuger wrote:
>>> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag community? I am aware that there are a lot
>>> of different issues to be solved to achieve the final library state, but this could arguably be a useful intermediate
>> I'd like to tell at least that I personally do favor the approach
>> described by Przemyslaw Iskra. IMHO it is the right way to go.
[snip]
> 
> I'd second that. Turning the jtag shell commands into API functions
> would keep the implications low since there are not many fundamental
> changes involved. It will be just the first of a series of steps
> towards a well-featured UrJTAG library, however.
> 
> My basic view on this topic is to first do the transition to liburjtag.a
> as shown in Przemyslaw's code.
[snip]
> Concerning the timeline, my idea is to release 0.10 soon. There's too
> much accumulated in SVN already and I point support requests to SVN
> check-outs too often in the meantime. Work on liburjtag won't make
> things less complicated.
>>From a personal perspective, I could support efforts for liburjtag best
> in the second half of April.

I must schedule my work on liburjtag mainly during April, too, and I 
wouldn't mind starting quickly.

Is your preference that we wait for the release of 0.10 and then 
integrate + extend the work by Przemyslaw? If so, when do you expect the 
0.10 release? Or could we just equally well create a branch from current 
trunk, and work on that?

Rutger

Arnim Läuger wrote:
>>> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag community? I am aware that there are a lot
>>> of different issues to be solved to achieve the final library state, but this could arguably be a useful intermediate
>> I'd like to tell at least that I personally do favor the approach
>> described by Przemyslaw Iskra. IMHO it is the right way to go.
[snip]
> 
> I'd second that. Turning the jtag shell commands into API functions
> would keep the implications low since there are not many fundamental
> changes involved. It will be just the first of a series of steps
> towards a well-featured UrJTAG library, however.
> 
> My basic view on this topic is to first do the transition to liburjtag.a
> as shown in Przemyslaw's code.
[snip]
> Concerning the timeline, my idea is to release 0.10 soon. There's too
> much accumulated in SVN already and I point support requests to SVN
> check-outs too often in the meantime. Work on liburjtag won't make
> things less complicated.
>>From a personal perspective, I could support efforts for liburjtag best
> in the second half of April.

I must schedule my work on liburjtag mainly during April, too, and I
wouldn't mind starting quickly.

Is your preference that we wait for the release of 0.10 and then
integrate + extend the work by Przemyslaw? If so, when do you expect the
0.10 release? Or could we just equally well create a branch from current
trunk, and work on that?

Rutger

------------------------------------------------------------------------------
_______________________________________________
UrJTAG-development mailing list
UrJTAG-development@...
https://lists.sourceforge.net/lists/listinfo/urjtag-development

>> Concerning the timeline, my idea is to release 0.10 soon. There's too
>> much accumulated in SVN already and I point support requests to SVN
>> check-outs too often in the meantime. Work on liburjtag won't make
>> things less complicated.
>>>> From a personal perspective, I could support efforts for liburjtag best
>> in the second half of April.
> 
> I must schedule my work on liburjtag mainly during April, too, and I
> wouldn't mind starting quickly.
> 
> Is your preference that we wait for the release of 0.10 and then
> integrate + extend the work by Przemyslaw? If so, when do you expect the
> 0.10 release? Or could we just equally well create a branch from current
> trunk, and work on that?

My intention was to have trunk stable until 0.10 release. I'm unclear
if renamed and moved files in the branch directory can be merged
back to trunk nicely. If this is feasible with SVN, then we could open
up a branch as soon as someone wants to begin with the work.

Arnim

Waschk,Kolja wrote:
>> Is the path as laid out by Przemyslaw considered a good idea by the UrJtag community? I am aware that there are a lot
>> of different issues to be solved to achieve the final library state, but this could arguably be a useful intermediate
>>     
>
> Hi,
>
> I'm sorry I haven't commented much in the past, I'm very busy at work
> and now will be on holiday until mid April, but on this important topic
> I'd like to tell at least that I personally do favor the approach
> described by Przemyslaw Iskra. IMHO it is the right way to go.
>
> I once planned to design the library API begining with the design of an
> efficient communication protocol, but as we know, that isn't done now
> even after almost two years. So, the logical alternative is to base the
> API design on what's already there in the code.
>
> There is a minor issue with libtool - with libtool, the build process
> can become very time consuming in Cygwin/MinGW environment. Switching to
> CMake would help a lot in this regard - but that's no priority and
> probably just a current personal preference.
>
> In the long term, the interface to Perl and other languages could be
> implemented using SWIG. While it certainly isn't perfect in every
> regard, it can be of great help.
>
> Kolja
>   

Yes, hurray for CMAKE!   We are using CMAKE successfully in Kicad.  In 
fact, I doubt Cygwin is even needed when using CMAKE, because there is 
no call for a bash shell.

SWIG for other languages such as Python or Java is also a good idea.

Good stuff, right direction.

Dick

Arnim Läuger wrote:
>> Rutger Hofman wrote:
>>> Is the path as laid out by Przemyslaw considered a good idea by
>>> the UrJtag community?

> Kolja Waschk wrote:
>> I'd like to tell at least that I personally do favor the approach 
>> described by Przemyslaw Iskra. IMHO it is the right way to go.

> I'd second that. Turning the jtag shell commands into API functions
> would keep the implications low since there are not many fundamental
> changes involved. It will be just the first of a series of steps
> towards a well-featured UrJTAG library, however.
> 
> My basic view on this topic is to first do the transition to liburjtag.a
> as shown in Przemyslaw's code. This includes redefining the command
> shell interface to have commands separated from API functionality. At
> the moment src/cmd/* implements shell commands AND related
> functionality, this needs to change then.
> Namespace clean-up and file renaming should happen here as well.
> 
> As a result, liburjtag would expose library functions that are
> equivalent to the currently existing commands. I'd call this the
> legacy/cmd API.

As a preparation for doing the Librarification, I would like to discuss 
any changes I will perform in some more detail. More messages with 
deeper detail will follow...

The work performed by Przemyslaw Iskra (as from his svn) seems to have 
focused mostly on using libtool, when I compare checkouts.

For starters, I propose to separate this Librarification from any other 
changes, like changing the build system, adding bindings for other 
languages etc. IMHO those changes are not an essential part of 
Librarification: they can be done separately without problems. 
Integrating them with Librarification may obfuscate the source of any 
problems, and it may hold back the work because of discussions involved 
:-) . Opinions?

List of proposed changes
========================

  - create one liburtjag in stead of various smaller libs;
  - create a public API for the library functionality; the 'shell' cmd 
is just one main program that should invoke the API;
  - add error handling for the public API functions;
  - where appropriate, move stuff from the 'shell' cmd into the library 
(while retaining all functionality for the parts database);
  - clean up the name space where conflicts may arise, by adding 
prefixes to:
     + global symbols
     + types
     + include file names
     + preprocessor symbols
  - rename source files.

Are there other things that should be refactored, like places where the 
code tree should be reorganized or whatever? Or is that best left as a 
separate step?

Plan of attack
==============

svn claims it can handle file renames and merging. I am not 100% 
confident that that really works. So, to be on the safe side, I would 
like to await release 0.10 and then create a branch. Would the 
repository need to be frozen while the Librarification is under way? 
Since svn can do branch merging across file renames, that would 
technically be unnecessary. And I think I will be the one who will merge 
the Librarification branch into trunk when all is over. Still, large 
changes on trunk would make the merge significantly more work. My 
proposal: no freeze, but encourage other commits to await the merge.

How should I commit on this Librarification branch? I like to commit 
frequently, even broken builds -- and sometimes that seems unavoidable 
when files are renamed. This means that that branch is unusable for 
other people, except for browsing. If other people do really need write 
access to a Librarification branch, I would propose to create a 'stable' 
Librarification branch beside my unstable working branch. FWIW, I am not 
in favor of 2 branches, because of the maintenance involved.

Issues to be discussed (1)
==========================

OK, now a (first) list of issues that should be discussed before I can 
even think of starting. Please, give your opinion.

What is a good project-identifying prefix?
------------------------------------------

I can think of:
  - urjtag_
  - jtag_
  - urj_
personally (FWIW) I prefer three-letter prefixes. So, for me, do urj_.

Symbol names
------------

In my projects (FWIW) I prefix global symbols by a project prefix and 
any number of appropriate module prefixes. This way, symbols from 
different modules can have the same 'basename' and still be exportable. 
E.g. I have mrg_acl_tag_locate(), mrg_acl_reader_locate(), 
mrg_util_sort_locate(), etc etc for things that _locate(). Because of 
this style, naming convention is 'inside-out': deeper submodules get 
appended, and the 'basename' comes last. For UrJTAG, that might make 
symbol names like urj_tap_cable_<function>().

Type names and preprocessor symbols
-----------------------------------

The changes to symbol names would also apply to type names. Besides, do 
we want to orthogonize type names, e.g. by having them uniformly end in 
_t, always having struct typedefs, etc?
Preprocessor symbols that we define should ideally also be 
disambiguated, like #define URJ_BSBIT_INPUT i.s.o. #define BSBIT_INPUT. 
FWIW, I am in favor of this change.

Coding style
------------

UrJTAG has a favorite coding style, GNU with -nut and -bli0. Lots of 
files have a different coding style. What do we want?
   + retain coding style of individual files, or
   + move to a uniform coding style (run indent over the source)
Changing to a uniform coding style will make changes harder to track. 
Anyhow, if it is desired, I think it is better to do this as a separate 
step, outside the Librarification.

What do we do with any warnings generated by the compiler? Should I fix 
any that I meet? Should we enable -Werror? FWIW, I have -Werror in my 
projects because I dislike warnings :-) Any other desirable warnings 
like -Wundef -Wformat -Wformat-nonliteral ...? Are we content with the C 
dialect (which currently appears to be GNU C), or would we prefer -ansi 
or -std=c99 or ...?
Or should I stay away from all this?

File names
----------

If we switch from multiple small libs to one liburjtag, then file names 
must be unique. A quick check (find | uniq -d) shows that the following 
names occur multiple times:
bsdl.c
cable.c
ChangeLog
.cvsignore
detect.c
detectflash.c
discovery.c
idcode.c
instruction.c
Makefile.am
part.c
README
readmem.c
register.c
signal.c
svf.c
svf.h
tap.c
writemem.c

By prefixing source file names with a prefix that reflects the 
module(s), source file names will become unique (ignoring pathological 
cases). So, that would mean: src/cmd/cable.c and src/tap/cable.c would 
become src/cmd/cmd_cable.c and src/tap/tap_cable.c, or 
src/cmd/urj_cmd_cable.c and src/tap/urj_tap_cable.c. Any preference?

Global API include file
-----------------------

I assume we want one global include file urjtag.h that specifies the 
complete library API. It will probably #include other public include 
files. Would it be a good idea to annotate the API in this top-level 
include file in Doxygen style? FWIW, I like Doxygen. It forces you to 
document your API and that's A Good Thing. Would we then want to 
Doxygen-annotate the other include files too? That will be a major, 
nonautomatable effort, which ideally would be carried out by the authors 
of the respective modules.

Error handling
--------------

How should the API functions handle errors? In current cmd, an error 
report is printed, and the cmd returns 1 for nonserious error or -1 for 
serious error. For a library, that is unsatisfactory. A library function 
should return more descriptive error status, possibly by setting 
global/glocal information. I have no ideas yet of a suitable set of 
errors. Opinions?

Public include file names
-------------------------

Do we want to rename all public include files too by adding a prefix, so 
there is less chance of a collision with include files from other 
software packages? We would have include/urj_cmd.h i.s.o. include/cmd.h, 
for example. FWIW, I am in favor of this change. Opinions?

Imported libs
-------------

Are there modules that are borrowed from elsewhere, which must escape 
this whole transformation? Else, we might lose the possibility to easily 
pull upgrades.

Rutger

Rutger, thanks for compiling this list of discussion topics!

> changing the build system, adding bindings [...]
> IMHO those changes are not an essential part of Librarification: they
> can be done separately without problems.

I agree. In my experience, transistion to CMake is no big deal at
any time if desired, so we do not have to think about that now. However,
it could make life easier before spending too much time on libtool if
that turns out to make problems here and there. Well, there haven't been
many(1) comments on CMake here, so first we should just use libtool
because it's better known to those who make the transition, right?

Adding bindings for other languages may require some changes to data
types and mechanisms later. Therefore we should now concentrate on
separation and renaming, but take care to change only as few types and
structures as possible, and postpone such optimization until later when
adding bindings.

> like to await release 0.10 and then create a branch. Would the

Now we have trunk/jtag, which contains the 0.x code. I'd copy it to
trunk/urjtag and do the librarification there, and later move "jtag"
out of the way. A trunk/README.jtag_vs_urjtag could tell code browsers
where to look for current code.

> proposal: no freeze, but encourage other commits to await the merge.

I encourage you to commit frequently in trunk/urjtag then, even broken
builds. We can consider creating a "stable" branch at any time if
someone becomes impatient, but I think we can get along without it for
some time.

> What is a good project-identifying prefix?

"jtag_" shouldn't be used as it is so generic. I prefer urjtag_ but
would be happy with just urj_ or even uj_ as well.

> symbol names like urj_tap_cable_<function>().

Ok

> Type names and preprocessor symbols

I'm personally no friend of pu_ and _t prefixes and tails.

> disambiguated, like #define URJ_BSBIT_INPUT i.s.o. #define BSBIT_INPUT. 
> FWIW, I am in favor of this change.

Ok

> + move to a uniform coding style (run indent over the source)
> Changing to a uniform coding style will make changes harder to track. 
> Anyhow, if it is desired, I think it is better to do this as a separate 
> step, outside the Librarification.

I'd really like to run indent over the source at some time. Now seems to
be quite a good moment to do that, because librarification will make it
hard to compare code before and afterwards anyway. But I'm unsure
whether it would be wise to do it before, during, or after the work..

> Should we enable -Werror? FWIW, I have -Werror in my

While I also like to eliminate all warnings, there will probably always
be a few on some systems. I think we should enable many warnings, but
not turn them into errors.

> file names must be unique

I believe it isn't necessary to require a common prefix per
subdirectory everywhere, but in the cmd/ directory it seems to be a good
idea to prefix command implementations with cmd_ as you wrote. That
probably would fix most collisions.

Another question is if filenames should be chosen in a way to avoid
collisions with application file names, exactly like function and
variable names are prefixed with urjtag_/urj_/... - is this required? I
can only think of problems when debugging, if the debugger doesn't care
about full paths when specifying source files.

> Global API include file

> I assume we want one global include file urjtag.h that specifies the 
> complete library API. It will probably #include other public include

Yes. Maybe a global "urjtag.h" that includes several "urjtag/*.h". That
way, I think no extra file name prefix is needed for include files.

> include file in Doxygen style? FWIW, I like Doxygen. It forces you to

Absolutely, yes.

> How should the API functions handle errors? In current cmd, an error

We have that common "chain" structure. An integer and string for
reporting errors could be placed in there, or even a linked list. We
also need a better way for printing messages even if there is no error.
Again, the "chain" structure could be used to transfer those messages,
either by just containing space (or pointers) for the text, or providing
a callback function like printf to be used by UrJTAG code.

> imported libs

AFAIK only lib/ contains snippets from elsewhere.

Kolja

Hi,

urjtag-development-bounces@... schrieb:
> I agree. In my experience, transistion to CMake is no big deal at
> any time if desired, so we do not have to think about that now. However,
> it could make life easier before spending too much time on libtool if
> that turns out to make problems here and there. Well, there haven't been
> many(1) comments on CMake here, so first we should just use libtool
> because it's better known to those who make the transition, right?

oh please don't use cmake. That's a pain for crosscompiling and doesn't improve
much apart from a better configuration performance.
What is the current issue with autotools?

Greetings

Florian

-- 
The dream of yesterday                  Florian Boor
is the hope of today                    Tel: +49 271-771091-15
and the reality of tomorrow.            Fax: +49 271-771091-19
[Robert Hutchings Goddard, 1904]        florian.boor@...
                                        http://www.kernelconcepts.de/en

Kolja wrote:
> Rutger Hofman wrote:
> 
>> changing the build system, adding bindings [...]
>> IMHO those changes are not an essential part of Librarification: they
>> can be done separately without problems.
> 
> I agree. In my experience, transistion to CMake is no big deal at
> any time if desired, so we do not have to think about that now. However,
> it could make life easier before spending too much time on libtool if
> that turns out to make problems here and there. Well, there haven't been
> many(1) comments on CMake here, so first we should just use libtool
> because it's better known to those who make the transition, right?

Should we switch to using libtool, then? You wrote before that libtool 
is undesirable under Cygwin/MinGW. We might also keep it the way it is 
for the time being (i.e. during Librarification).

> Now we have trunk/jtag, which contains the 0.x code. I'd copy it to
> trunk/urjtag and do the librarification there, and later move "jtag"
> out of the way. A trunk/README.jtag_vs_urjtag could tell code browsers
> where to look for current code.

Is your idea that 'vanilla' jtag/ keeps in existence beside jtag/ for a 
grace period? Then patches would need to be applied twice.

If we don't want to keep jtag/ alive, we might just as well branch/merge 
for the Librarification. My feeling is that svn might be slightly more 
helpful in doing the merge w/ changes that have occurred during 
Librarification in trunk/. This makes me tentatively prefer using 
branch/merge.

>> + move to a uniform coding style (run indent over the source)
>> Changing to a uniform coding style will make changes harder to track. 
>> Anyhow, if it is desired, I think it is better to do this as a 
>> separate step, outside the Librarification.
> 
> I'd really like to run indent over the source at some time. Now seems to
> be quite a good moment to do that, because librarification will make it
> hard to compare code before and afterwards anyway. But I'm unsure
> whether it would be wise to do it before, during, or after the work..

Well, if we do it now, I prefer to start Librarification w/ running 
indent. That way, I can set editor configs consistently during the 
editing process.

>> file names must be unique
> 
> I believe it isn't necessary to require a common prefix per
> subdirectory everywhere, but in the cmd/ directory it seems to be a good
> idea to prefix command implementations with cmd_ as you wrote. That
> probably would fix most collisions.
> 
> Another question is if filenames should be chosen in a way to avoid
> collisions with application file names, exactly like function and
> variable names are prefixed with urjtag_/urj_/... - is this required? I
> can only think of problems when debugging, if the debugger doesn't care
> about full paths when specifying source files.

Another example that I met was w/ a broken ar. I had to unpack libraries 
and use the .o files from it. This actually led to conflicts (two files 
malloc.o from different libraries). I agree that this is exceptional.

It would be more consistent if we have file names carry a prefix not 
only in cmd/ but everywhere. However, doing it only in cmd/ solves 
conflicts (for now), so if the list has a preference for doing it only 
in cmd/, I'm OK.

>> How should the API functions handle errors? In current cmd, an error
> 
> We have that common "chain" structure. An integer and string for
> reporting errors could be placed in there, or even a linked list. We

OK. So the mechanism will be that functions return an error value (like 
-1 or NULL) and register error info in chain->error, and the caller must 
inspect/reset the error info in chain->error. About one error, or a list 
a la OpenSSL: my first intuition is to do just one error, but to set it 
up in a way that allows easy changes by using wrapper functions i.s.o. 
having code access the chain->error struct directly.

> also need a better way for printing messages even if there is no error.
> Again, the "chain" structure could be used to transfer those messages,
> either by just containing space (or pointers) for the text, or providing
> a callback function like printf to be used by UrJTAG code.

OK, I will add a vprintf-type routine to chain-> that is called by 
urj_printf(const char *fmt, ...). Should it default to printing on 
stdout or on stderr?

> Kolja
> 

Rutger

> oh please don't use cmake. That's a pain for crosscompiling
> and doesn't improve much apart from a better configuration
> performance.

OpenEmbedded has a cmake.bbclass, so there it's quite simple. 
Some packages use that already, e.g. taglib, cdrkit, strigi, 
sakura, gizmod, musicbrainz, gammu, wt, opensync ...

So, with the right tools cross-compiling with cmake is a pice of 
cake :-)

Hi,

> Should we switch to using libtool, then?

IMHO yes. After all, we're currently using autotools and shouldn't
change too much at once. But we should experiment with CMake at a later
time. I too found it performed well _especially_ for cross compiling
(well, in a mixed i686-Linux/bfin-uClinux/MinGW project).

> You wrote before that libtool is undesirable under Cygwin/MinGW.

I experienced problems with passing flags to the linker when I wanted to
use Cygwin GCC in MinGW mode (-mno-cygwin), and it seemed to work quite
slow overall because it does many file accesses when scanning
dependencies and building libraries. But these problems might have been
caused in particular due to my attempt to use convenience libraries
(objects from subdirectories each put into separate library files, later
merged into one larged library). Let's not use convenience libraries but
simply build one single liburjtag from all sources, as you suggested.

> Is your idea that 'vanilla' jtag/ keeps in existence beside jtag/ for a 
> grace period? Then patches would need to be applied twice.

I thought it might be polite to keep 0.10++ in existence, open only for
bugfixes, until we feel that users should be talked into switching to 1.0
(library) if they encounter problems with 0.10. I.e. until 1.0 is stable.

> in doing the merge w/ changes that have occurred during Librarification in 
> trunk/. This makes me tentatively prefer using branch/merge.

A SVN copy of jtag/ named urjtag/ actually is just a branch, although it
doesn't reside in branches/ but in trunk/. Technically that doesn't
matter. Or maybe I misunderstood you?

> Well, if we do it now, I prefer to start Librarification w/ running indent. 
> That way, I can set editor configs consistently during the editing process.

Right, that's a good point. What do the others think about this?

> It would be more consistent if we have file names carry a prefix not only in 
> cmd/ but everywhere. However, doing it only in cmd/ solves conflicts (for 
> now), so if the list has a preference for doing it only in cmd/, I'm OK.

I'd say keep file names short as before, then more detailed if necessary
just to avoid collisions.

> a way that allows easy changes by using wrapper functions i.s.o. having code 
> access the chain->error struct directly.

Yes, good idea.

> OK, I will add a vprintf-type routine to chain-> that is called by 
> urj_printf(const char *fmt, ...). Should it default to printing on stdout or 
> on stderr?

How about introducing a (simple) "log level"? Maybe

0 all     - every single bit as it is transmitted
1 comm    - low level communication details
2 debug   - more details of interest for developers
3 detail  - verbose output
4 normal  - just noteworthy info
5 errors  - only fatal errors

Kolja

>> Is your idea that 'vanilla' jtag/ keeps in existence beside jtag/ for a 
>> grace period? Then patches would need to be applied twice.
> 
> I thought it might be polite to keep 0.10++ in existence, open only for
> bugfixes, until we feel that users should be talked into switching to 1.0
> (library) if they encounter problems with 0.10. I.e. until 1.0 is stable.

I'm indecisive on this one. I like the idea of having a space where
issues can be fixed as Kolja pointed out. Applying patches to both bases
adds some effort and gets probably annoying, yes. But this isn't my main
concern here.
In case users don't follow to urjtag branch we'd probably lose
feedback on its current quality level. IMHO testing the whole codebase
can't be achieved under lab conditions. At least for me there are
features that I can't test at all. Extending such a period might end in
a stable 1.0 with low quality.
In the end I guess the 0.10++ branch shouldn't live for too long.

>> Well, if we do it now, I prefer to start Librarification w/ running indent. 
>> That way, I can set editor configs consistently during the editing process.
> 
> Right, that's a good point. What do the others think about this?

Agree.
Is there a link between indent and indentation modes of common editors?
E.g. emacs, vim?

>> It would be more consistent if we have file names carry a prefix not only in 
>> cmd/ but everywhere. However, doing it only in cmd/ solves conflicts (for 
>> now), so if the list has a preference for doing it only in cmd/, I'm OK.
> 
> I'd say keep file names short as before, then more detailed if necessary
> just to avoid collisions.
> 
>> a way that allows easy changes by using wrapper functions i.s.o. having code 
>> access the chain->error struct directly.
> 
> Yes, good idea.
> 
>> OK, I will add a vprintf-type routine to chain-> that is called by 
>> urj_printf(const char *fmt, ...). Should it default to printing on stdout or 
>> on stderr?
> 
> How about introducing a (simple) "log level"? Maybe

We had some thoughts on this earlier, see
http://sourceforge.net/tracker/?func=detail&aid=1958007&group_id=193266&atid=944728

Waschk,Kolja wrote:
>> Should we switch to using libtool, then?
> 
> IMHO yes.

OK.

>> in doing the merge w/ changes that have occurred during 
>> Librarification in trunk/. This makes me tentatively prefer using 
>> branch/merge.
> 
> A SVN copy of jtag/ named urjtag/ actually is just a branch, although it
> doesn't reside in branches/ but in trunk/. Technically that doesn't
> matter. Or maybe I misunderstood you?

You are right of course. The major difference is that anybody who checks 
out trunk/ will also see urjtag/. So, I will start my work by adding a 
README in urjtag/ that scares users away until the Librarification is 
finished.

>> OK, I will add a vprintf-type routine to chain-> that is called by 
>> urj_printf(const char *fmt, ...). Should it default to printing on 
>> stdout or on stderr?
> 
> How about introducing a (simple) "log level"? Maybe
> 
> 0 all     - every single bit as it is transmitted
> 1 comm    - low level communication details
> 2 debug   - more details of interest for developers
> 3 detail  - verbose output
> 4 normal  - just noteworthy info
> 5 errors  - only fatal errors

And this should still invoke a registered vprintf-type routine. What do 
you people think, should this by default print to stdout or stderr, or 
should this depend on the log level? I.e. do 0..4 on stdout and 5 on 
stderr? We would need 2 registered vprintf-type routines then, one for 
'regular' logging and one for 'fatal' logging. FWIW, I like the idea of 
a pluggable log/print routine: that way, it is easy to direct logging to 
a file or buffer. In embedded systems, that's a goodie.

Indentation
===========

Another thingy. I took a look at the preferred coding style, GNU with 
-nut and -blio0, by applying it to one of the files. The indentation 
unit is 2 spaces. OK, I am aware that we will now enter religion 
country, but here goes! The result looks cramped to my eyes. I would 
plead in favor of a larger indentation unit like 4 or 8 spaces.

Pros re increasing the indentation unit:
  - vanilla GNU coding style in fact indents 4 spaces: first 2 spaces 
for the '{', then another 2 spaces for the code;
  - Linux kernel style has 8 spaces; Sun Java coding style has 4 spaces; 
I know of no other projects that have fewer than 4;
  - 4 or 8 spaces gives significantly improves readability, IMHO;
  - FWIW, current code has 8 spaces (= 1 tab) nearly everywhere; I think 
there are only about 2 files that follow the current recommendation.

Cons:
  - you already advertised 2 spaces on 
http://urjtag.wiki.sourceforge.net/Development so I assume you really 
like it.

Happy worshipping,

Rutger

Hi,

> And this should still invoke a registered vprintf-type routine. What do you 
> people think, should this by default print to stdout or stderr, or should

Ok, sorry, I didn't actually reply to your question. Yes, something like
errors on stderr, others on stdout would be fine. stderr should get
those messages that are meant to be displayed even to the user (or log)
who redirected stdout to /dev/null ("no output if all is well").

> spaces. OK, I am aware that we will now enter religion country, but here 
> goes! The result looks cramped to my eyes. I would plead in favor of a larger

4 is okay for me too. I wasn't aware it is so uncommon.

Kolja

Waschk,Kolja wrote:
> Hi,
>
>   
>> And this should still invoke a registered vprintf-type routine. What do you 
>> people think, should this by default print to stdout or stderr, or should
>>     
>
> Ok, sorry, I didn't actually reply to your question. Yes, something like
> errors on stderr, others on stdout would be fine. stderr should get
> those messages that are meant to be displayed even to the user (or log)
> who redirected stdout to /dev/null ("no output if all is well").
>
>   
>> spaces. OK, I am aware that we will now enter religion country, but here 
>> goes! The result looks cramped to my eyes. I would plead in favor of a larger
>>     
>
> 4 is okay for me too. I wasn't aware it is so uncommon.
>
> Kolja
>   

This is exciting stuff.


I recently added CMake support to OpenOCD, but the turds over there have 
yet to accept it and I am frustrated with that whole group of power 
hungry, C language ruining hacks.   I don't understand it, but I think 
that if you write enough embedded code, you end up thinking that all 
code should look that same way.  The guys were using hex constants 
instead of meaningful defines, etc.  The politics and resistance to 
change are crippling the project.


I am a lead developer on Kicad and have owned a software company for 26 
years so I should not have to beg to get my work accepted.   My 
customers pay me $200/hour for my work.  So what gives, I have to beg to 
get code into an open source project?    That is not right, and I won't 
do it, my time is too valuable.


By comparison to OpenOCD, the Kicad project is grease lightening.  We 
move and change fast.


My humble suggestions, from 26 years of experience:


1) Use uncrustify, not indent for the beautification.  I did a detailed 
search for beautifiers 2 years ago.  Uncrustify wins.   Grab the 
uncrustify.cfg file from Kicad's repo.  I wrote it myself, it will work 
on C or C++.   Uncrustify's author will respond to bug reports, and is a 
fine guy to lean on for this kind of tool.


2) Switch to CMake *as you do* the retransformation, not after.  Skip 
libtool, it is obsoleted by CMake as far as I can tell.  And run away 
from autotools immediately.  With CMake you can build from the ground 
up, a few source files at a time.  You could build a static libarary 
during the transformation, and then at the end switch to a dynamic target.


3) Seriously consider switching to C++ soon, perhaps during the 
transformation.   It is reasonably easy to get C code to compile with 
C++, and that gives you the option of adding exception support later.  
It is not impossible to have code that compiles with either C or C++, 
but that need not be a goal.  The vision I sense here from you guys is 
that you want bindings to other languages, and that suggests a minimum 
host platform, so C++ can be expected there.


I really think you guys have a nice plan of attack.  I realize you don't 
know me.  But I do have the stated experience, and with that comes an 
understanding of what works and what works less well.


If there is anything I can help with, let me know.


I hope this helps,

Dick Hollenbeck

> Switch to CMake [...], C++ [...]

Dick,

the UrJTAG development proceeds in small steps. We appreciate when
someone finds more time to do some major overhaul like the upcoming
"Librarification". It is important to let the developers use the tools
they prefer personally. As far as I can see that's libtool now, not CMake.

UrJTAG may switch to CMake at some time in the future, when there's a 
stable code base again. It's not ruled out now. Maybe we just give it a
try at some time, in parallel to libtool.

UrJTAG will not use C++ at anytime soon, though. I've seen too many
embedded projects where C++ was tried but the (same!) developers
couldn't actually make good use of it, and worse, the ability to relate
written code with actual low level activity on the target was lost.
This lead to bad performance in every regard. And, as I wrote in an
earlier post on this topic, I consider UrJTAG to be an embedded project,
at least the library part of it.

Kolja

Waschk,Kolja wrote:
>> Switch to CMake [...], C++ [...]
>
> Dick,
>
> the UrJTAG development proceeds in small steps. We appreciate when
> someone finds more time to do some major overhaul like the upcoming
> "Librarification". It is important to let the developers use the tools
> they prefer personally. 

Exactly.

> As far as I can see that's libtool now, not CMake.

That is not clear.   It could be clear if somebody were to actually try 
CMake and then state a dislike for it.

>
> UrJTAG may switch to CMake at some time in the future, when there's a 
> stable code base again. It's not ruled out now. Maybe we just give it a
> try at some time, in parallel to libtool.
>
> UrJTAG will not use C++ at anytime soon, though. I've seen too many
> embedded projects where C++ was tried but the (same!) developers
> couldn't actually make good use of it, and worse, the ability to relate
> written code with actual low level activity on the target was lost.
> This lead to bad performance in every regard.

C++ can be "over used", agreed.  It can complicate things if abused, but 
it does not have to if properly used.  If properly used it can be an 
advantage:


** type safe linkage
** object oriented
** exceptions
** automatic destruction
** inheritance
** interfaces


When you start to get into operator overloading and using fat class 
libraries that do too much hiding, then you get into some areas where 
the language can start to work against you.  But on balance, for an 
application (not an OS or OS level driver), that is for user space type 
code, C++ is superior to C in my opinion.  And this is not a minority 
point of view. 

I think where the disagreement comes into play is: what is UrJTAG?  Is 
it a userspace application, or is is an embedded driver that needs to 
run on a match stick or pin head?


I only want to help the project, because like all projects, there is 
never enough qualified help.   Maximizing the productivity of the 
qualified contributors is important, because their work will be more 
rewarding and substantial in output.  CMake is such a tool that can 
increase that, as is C++.


> And, as I wrote in an
> earlier post on this topic, I consider UrJTAG to be an embedded project,
> at least the library part of it.

That is not a view that I share.  I view it as a library or application 
which runs on a substantial computer.  I would never need to run it on a 
match stick, only on a desktop PC.

In the end, those that do the work will get to decide, so my input is 
there to consider, beyond that I will  not lose any sleep over it.


Dick

Rutger Hofman wrote:
> As a preparation for doing the Librarification, I would like to discuss 
> any changes I will perform in some more detail. More messages with 
> deeper detail will follow...

The first API issue that I would like to discuss concerns the "cable" 
command and its invocation of cable->driver->connect(++params). 
Transformation into a straight API with function arguments in place of a 
command arguments fail here, because of the diversity of connect() 
arguments. This is further complicated because the generic USB and 
parallel connect routines optionally recognize 4 switches (that should 
become four fixed-place parameters), and then dispatch to driver 
connect() calls that take variable arguments again.

I see any number of alternatives for a connect() API implementation; let 
me mention:
a) for this variable-number argument function connect(), keep params as 
an array of const char * with trailing NULL: i.e. nothing changes;
b) make this a complete varargs call:
      connect(cable, va_list ap);
    and each cable driver knows which arguments to expect at which 
places wrapped into ap. I'm a bit suspicious whether varags are always 
handled correctly through function calls, i.e. for further calling into 
driver connect()s;
c) provide an interface that resembles X11 API programming: there is an 
argument struct into which (key, value) pairs are assembled one by one. 
This resembles a) a lot.

I don't particularly like any of these. Is there an obvious good 
solution that I overlook? Or, if not, what is your opinion?

Rutger

Dick Hollenbeck wrote:
> Waschk,Kolja wrote:
>>> Switch to CMake [...], C++ [...]

[snip CMake discussion]

>> UrJTAG will not use C++ at anytime soon, though. I've seen too many
>> embedded projects where C++ was tried but the (same!) developers
>> couldn't actually make good use of it, and worse, the ability to relate
>> written code with actual low level activity on the target was lost.
>> This lead to bad performance in every regard.
> 
> C++ can be "over used", agreed.  It can complicate things if abused, but 
> it does not have to if properly used.  If properly used it can be an 
> advantage:

[snip]

I, for starters, want to use liburjtag in my embedded project. There is 
not much use for C++ here because eCos doesn't provide the C++ libs. So, 
I am in favour of sticking to C.

> I only want to help the project, because like all projects, there is 
> never enough qualified help.   Maximizing the productivity of the 
> qualified contributors is important, because their work will be more 
> rewarding and substantial in output.  CMake is such a tool that can 
> increase that, as is C++.

Qualified help and feedback are of huge value, of course!

>> And, as I wrote in an
>> earlier post on this topic, I consider UrJTAG to be an embedded project,
>> at least the library part of it.
> 
> That is not a view that I share.

See above.

Rutger

Hi Rutger,

I wouldn't choose the varargs, it might be complicated to model this for
use from other languages. The key/value pairs seem to be best IMHO. The
cable code wouldn't have to scan for '=' in strings and the calling
code. if it already has config data in key/value pairs, wouldn't have to
paste a '=' between.

Another solution could be not to provide an one-for-all "cable" setup
function but instead allow / leave it to the app to call cable-specific
code directly for configuration, and to fill out the cable-pointers in
"chain".

Kolja

Hi Dick,

>> As far as I can see that's libtool now, not CMake.
> That is not clear.   It could be clear if somebody were to actually try CMake 
> and then state a dislike for it.

You're right, it's not clear. It's just that autotools are the current
tool of choice and libtool seems to me the obvious extra to be tried first.

I work on one project where I dumped libtool, because I wasn't able to
compile the project with the -mno-cygwin flag. That might be the case
here, too. CMake was the solution. On the other hand, CMake can't do
some things correctly as we do them now with autotools, e.g. the SVN
detection. And all project-specific detection of external libraries,
ftdi, ftd2xx, libusb/-win32, inpout32, ioperm etc. would have to be
rewritten.

Kolja

Dick Hollenbeck wrote:
> 1) Use uncrustify, not indent for the beautification.  I did a detailed 
> search for beautifiers 2 years ago.  Uncrustify wins.   Grab the 
> uncrustify.cfg file from Kicad's repo.  I wrote it myself, it will work 
> on C or C++.   Uncrustify's author will respond to bug reports, and is a 
> fine guy to lean on for this kind of tool.

The current function call/definition style nearly everywhere in UrJTAG 
now has:

int fun( int arg )
{
}

x = fun( a );

but the GNU style is:

int fun (int arg)
{
}

x = fun (a);

Do we actually want to change to GNU style here? FWIW, I am neutral (I 
prefer: x = fun(a);), although I think GNU style unappealing in:

     printf (_ ("Fatal error"));

If we prefer current practice to GNU style: I seem unable to find an 
indent option for this. OTOH, uncrustify supports it.

One thing that didn't make me love uncrustify immediately: I did apt-get 
install uncrustify on my Ubuntu Hardy system, and it dumps core during a 
call uncrustify --show-config. apt-get uncrustify is 0.41; bleeding edge 
is 0.52, and it doesn't dump.

Rutger

> Do we actually want to change to GNU style here? FWIW, I am neutral (I
> prefer: x = fun(a);), although I think GNU style unappealing in:

I'm also neutral. The goal is to use some style that is supported by as
many editors/tools as possible without much configuration effort. Or,
that's why I proposed indent, a style that is supported by at least one
tool that is almost omnipresent. indent seemed to be such tool. I didn't
know about uncrustify, that means it's certainly not omnipresent ;)

Hello,

in message <alpine.DEB.2.00.0904201245520.7834@...> Kolja Waschk wrote:
> > Do we actually want to change to GNU style here? FWIW, I am neutral (I
> > prefer: x = fun(a);), although I think GNU style unappealing in:
> 
> I'm also neutral. The goal is to use some style that is supported by as
> many editors/tools as possible without much configuration effort. Or,
> that's why I proposed indent, a style that is supported by at least one
> tool that is almost omnipresent. indent seemed to be such tool. I didn't
> know about uncrustify, that means it's certainly not omnipresent ;)

Indeed.

BTW - indent is highly configurable; for example, see what Linux uses
in the scripts/Lindent script. For the U-Boot project, we use
something like "Lindent -pcs"

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@...
Cogito cogito ergo cogito sum - "I think that I  think,  therefore  I
think that I am."          - Ambrose Bierce, "The Devil's Dictionary"

Wolfgang Denk wrote:
> Hello,
> 
> in message <alpine.DEB.2.00.0904201245520.7834@...> Kolja Waschk wrote:
>>> Do we actually want to change to GNU style here? FWIW, I am neutral (I
>>> prefer: x = fun(a);), although I think GNU style unappealing in:
>> I'm also neutral. The goal is to use some style that is supported by as
>> many editors/tools as possible without much configuration effort. Or,
>> that's why I proposed indent
> 
> Indeed.
> 
> BTW - indent is highly configurable; for example, see what Linux uses
> in the scripts/Lindent script. For the U-Boot project, we use
> something like "Lindent -pcs"

Right. Except that it cannot do the current layout in UrJTAG:

     printf( _( "Horrors\n" ) );

Indent lets us choose between -pcs:

    printf (_ ("Horrors\n"));

and -npcs:

    printf(_("Horrors\n"));

FWIW, I use -npcs in my projects; I think -pcs looks unappealing in the 
case of the _( ) macro. So, please choose.

Rutger

Rutger Hofman wrote:
> Dick Hollenbeck wrote:
>   
>> 1) Use uncrustify, not indent for the beautification.  I did a detailed 
>> search for beautifiers 2 years ago.  Uncrustify wins.   Grab the 
>> uncrustify.cfg file from Kicad's repo.  I wrote it myself, it will work 
>> on C or C++.   Uncrustify's author will respond to bug reports, and is a 
>> fine guy to lean on for this kind of tool.
>>     
>
> The current function call/definition style nearly everywhere in UrJTAG 
> now has:
>
> int fun( int arg )
> {
> }
>
> x = fun( a );
>
> but the GNU style is:
>
> int fun (int arg)
> {
> }
>
> x = fun (a);
>
> Do we actually want to change to GNU style here? FWIW, I am neutral (I 
> prefer: x = fun(a);), although I think GNU style unappealing in:
>
>      printf (_ ("Fatal error"));
>
> If we prefer current practice to GNU style: I seem unable to find an 
> indent option for this. OTOH, uncrustify supports it.
>
> One thing that didn't make me love uncrustify immediately: I did apt-get 
> install uncrustify on my Ubuntu Hardy system, and it dumps core during a 
> call uncrustify --show-config. apt-get uncrustify is 0.41; bleeding edge 
> is 0.52, and it doesn't dump.
>
> Rutger
>   

Because the uncrustify author is fairly responsive to bug reports, and 
because I was even asking for feature enhancements to which he was 
quickly responding, I ended up using a SVN "working copy" of the latest 
code.   The support is good, the speed of response to enhancement 
requests is good, and I did not see where indent was any match for it.

The bug you saw could be blamed on the debian packager, who knows.  He 
should have tested that, no?

Dick

On Mon, Apr 20, 2009 at 11:06:26AM +0200, Waschk,Kolja wrote:

> Another solution could be not to provide an one-for-all "cable" setup
> function but instead allow / leave it to the app to call cable-specific
> code directly for configuration, and to fill out the cable-pointers in
> "chain".

Right now "cable" is completely automatic, which may not be desirable
for a library. There should at least be some way to specify which device
to use, instead of scanning all the usb devices or parallel ports for a
specyfic identifier. Programmer may already know where the device is
connected. Think about cases where two or more usb cables of same type
are connected.

Annother thing to consider is that people build and want to use their
own cables. It would be nice to be able at least to test it without
having to integrate the code into the urjtag library. (I will have to
create a TCP based one, will tell you how it turns out.)

Best regards.

-- 
 ____  Sparky{PI] -- Przemyslaw _  ___  _  _  ........... LANG...Pl..Ca..Es..En
/____) ___  ___  _ _ || Iskra  |  | _ \| |  | : http://www........ppcrcd.pld-linux.org
\____\| -_)'___| ||^'||//\\// <   |  _/| |  | : JID......sparky<at>jabberes.org
(____/||   (_-_|_||  ||\\ ||   |_ |_|  |_| _| : Mail....sparky<at>pld-linux.org

Rutger Hofman wrote:
> Rutger Hofman wrote:
>> As a preparation for doing the Librarification, I would like to 
>> discuss any changes I will perform in some more detail. More messages 
>> with deeper detail will follow...

I'm starting the Grand Renaming Offensive, first in a quasi-automatic 
way. First questions, on #define-s:

1. Any objection if I remove the following macros from include/bus_driver.h:

#define CHAIN                   bus->chain
#define PART                    bus->part
#define INITIALIZED             bus->initialized

This implicit assumption of a local variable 'bus' is considered 
not-so-good programming practice -- and I agree w/ received wisdom here. 
I can do a global replace s/CHAIN/bus->chain/ etc, IMHO without loss of 
program readability.

2. Any objection if I make all macro names uppercase, at least in the 
include files?

3. Is it OK if I do small pedantic changes wherever I stumble into an 
occasion? Like, in the same file include/bus_driver.h, I would like to 
change:
    #define bus_printinfo(bus)      bus->driver->printinfo(bus)
into:
    #define bus_printinfo(bus)      (bus)->driver->printinfo(bus)

Rutger

Rutger Hofman wrote:
> Rutger Hofman wrote:
>> As a preparation for doing the Librarification, I would like to 
>> discuss any changes I will perform in some more detail. More messages 
>> with deeper detail will follow...

What is the value we attach to having the

    /* $Id: <filename> <number> <date> <committer> */

lines? I think this was very handy long ago, but my feeling nowadays is 
that it only generates false hits in svn diff.

Is it OK to remove these lines?

Rutger

> 1. Any objection if I remove the following macros from include/bus_driver.h:
> 
> 2. Any objection if I make all macro names uppercase, at least in the 
> include files?
> 
> 3. Is it OK if I do small pedantic changes wherever I stumble into an 
> occasion?

I'm fine with all of them.

Arnim

> Right now "cable" is completely automatic, which may not be desirable
> for a library. There should at least be some way to specify which device
> to use, instead of scanning all the usb devices or parallel ports for a
> specyfic identifier.

For FTDI based cables this is not entirely possible. Only libftdi offers
a function to open by a usb_device descriptor (ftdi_open_dev). libftd2xx
has nothing comparable and either looks for VID/PID/DESC (FT_Open) or an
incrementing number (FT_OpenEx).

This could be implemented as a new parameter to cable like
dev=/dev/bus/usb/... which won't be supported by libftd2xx.

> Programmer may already know where the device is
> connected. Think about cases where two or more usb cables of same type
> are connected.

Again for FTDI based cables: Users need to specify the product name or
serial number strings in this case. Both libftdi and libftd2xx select
the device based on this specific identification.
Don't know about the status with libusb based cable drivers here.


Arnim

Hi Rutger,

Re pcs/npcs:

I'd accept the unfortunate space between _ and ( and go with -pcs, which
is the default mode of indent, isn't it?

Re renaming offensive: I'm fine with all the changes you suggest

Re Id tag: It is very valuable when files appear somewhere out of
context, e.g. someone posts just a modified file to a forum or attaches
it to a tracker item. Please keep it in place.

Kolja

Arnim Läuger wrote:
>>> OK, I will add a vprintf-type routine to chain-> that is called by 
>>> urj_printf(const char *fmt, ...). Should it default to printing on stdout or 
>>> on stderr?
>> How about introducing a (simple) "log level"? Maybe
> 
> We had some thoughts on this earlier, see
> http://sourceforge.net/tracker/?func=detail&aid=1958007&group_id=193266&atid=944728

When doing logging/error reporting, it would be good to include file 
name, (function name,) line number in the report. That means that the 
logging function must be a macro (that calls a function with (among 
others) arguments __FILE__, __FUNCTION__, __LINE__).

Now, as we'd prefer to have the logger to accept variable arguments, we 
have two choices:

(1) make this a varags macro:
       URJ_LOG(log_level, const char *fmt, arg0, arg1, ...)

(2) use the parenthesizing technique:
       URJ_LOG(log_level, (const char *fmt, arg0, arg1, ....))
     which would essentially expand like
       #define URJ_LOG(level, msg) do { urj_log_print msg; } while (0)
     note the absence of parentheses in the expansion

(1) is a GCC extension.
(2) is portable but more cumbersome as each call must have extra 
parentheses.

Which one is preferable?

Rutger

Hi Rutger,

> (1) make this a varags macro:
>       URJ_LOG(log_level, const char *fmt, arg0, arg1, ...)
> (2) use the parenthesizing technique:
>       URJ_LOG(log_level, (const char *fmt, arg0, arg1, ....))
...
> Which one is preferable?

The second one is IMHO preferable because it's more portable.

Kolja

Rutger Hofman wrote:
> (1) make this a varags macro:
> (1) is a GCC extension.

Mwah, sorry, it's C99. FWIW, we already use C99 features like //-style 
comments.

Rutger

Status update:
==============

I committed a version that uses libtool to create one shared library, 
and that links with jtag.o to form executable jtag.

In preparation, global symbols have been prefixed with urj_<module>*, 
defines, typedefs, enums were also converted to be more 
namespace-friendly. I ran indent over the code base (-nut -bli0 -i4) 
(multiple times, as these things go...).

Lots of refactoring remain. Highlights:
- make a suitable API. I hope there will be lots of input from you, 
list, because I am not an UrJTAG expert
- lift stuff in the cmd implementations to the appropriate libraries, 
and have the cmd implementations invoke this through the API
- error handling
- logging

My plan is to attack error handling first, and then one by one convert 
the stuff in the cmd implementations. I will consult the list whenever 
it becomes interesting. Extensions of the API will follow w/ each 
conversion. Later on, we will also need to discuss how much of the API 
actually needs to be exported, and how much turns out to be for internal 
use of the library itself. IMHO, include files internal to the library 
should, preferentially, not be exported to the user of the library and 
its API.

For now, discussion of error handling
=====================================

>> How should the API functions handle errors? In current cmd, an error
> 
> We have that common "chain" structure. An integer and string for
> reporting errors could be placed in there, or even a linked list.

I previously reported that we could either use a varags macro that would 
allow file/line/function info to be registered w/ the error, or we could 
use a tric like:
   urj_error_set(urj_errno, chain, ("Here comes a %s list", "printf-style"))
But on better looks, I think this second approach doesn't work. It 
depends on calling:
    some_printf msg;
(where msg is ("Here comes a %s list", "printf-style") -- watch the 
(absence of) parentheses).
But if we want to store the error msg into the chain->error structure, 
we need to pass at least a pointer to the urj_chain_t. I cannot find a 
way to do that with the parentheses trick!
So, choices would now seem to be:
a) use a varargs macro
    solves all problems but might not be available
b) use a varags function
    we cannot get at __FILE__ __FUNCTION__ __LINE__
c) use a global/glocal struct variable for error handling, not a part of 
the chain struct. Then we can use the parentheses trick again, because 
the printer routine knows about the global/glocal struct.

Note: if we choose a) and configure finds out we don't have varargs 
macros, the implementation can use b), a varargs function -- with loss 
of functionality.

Any preferences? As I said before, we do already depend on c99 features 
like '//' comments. That attracts me to solution a) with b) as fallback.

If the list is interested, I can post a sample implementation/invocation 
for a) so we can take a good close look.

Rutger

Rutger Hofman wrote:
> - lift stuff in the cmd implementations to the appropriate libraries, 
> and have the cmd implementations invoke this through the API

About the instruction command: it has 3 flavors:
   instruction <INSTRUCTION>     # change active instruction for a part
   instruction length <LENGTH>   # change instruction length
   instruction <INSTRUCTION> <CODE> <REGISTER> # define new instruction

Question about the third flavor: is there a bound on the number of bits 
in <CODE> ? Is there a preferred data structure to pass in the code to 
the API library call?

--R

> About the instruction command: it has 3 flavors:
>    instruction <INSTRUCTION>     # change active instruction for a part
>    instruction length <LENGTH>   # change instruction length
>    instruction <INSTRUCTION> <CODE> <REGISTER> # define new instruction
> 
> Question about the third flavor: is there a bound on the number of bits 
> in <CODE> ?

Number of bits in <CODE> must equal <LENGTH>. This implies that the
second flavor was executed before the first invocation with the third
flavor. Violations will result in an error at runtime.

> Is there a preferred data structure to pass in the code to
> the API library call?

IMO, there are two options of data types that are already used
internally: a char string consisting of '0' and '1' or an integer array
containing 0 and 1. They differ also in the endianess. First bit applied
to TDI is index 0 for the integer array and index [len-1] for the
string.

The string is mainly used when interfacing with the user or standardized
file formats (SVF, BSDL) for two reasons:
a) these interfaces deal with strings
b) endianess of the string type matches the convention of the JTAG
   standard(s)

I'd propose to continue here (and for dr command) with the string type
mainly for reason b).

Arnim

Arnim Läuger wrote:
>> About the instruction command: it has 3 flavors:
>>    instruction <INSTRUCTION> <CODE> <REGISTER> # define new instruction
>> Is there a preferred data structure to pass in the code to
>> the API library call?
[snip]
> I'd propose to continue here (and for dr command) with the string type
> mainly for reason b).

OK, I'll do that.

--R

> Status update:
> ==============
> 
> I committed a version that uses libtool to create one shared library, 
> and that links with jtag.o to form executable jtag.

That's great news, thanks a lot for your work and dedication!

> Lots of refactoring remain. Highlights:
> - make a suitable API. I hope there will be lots of input from you, 
> list, because I am not an UrJTAG expert

This will require some brain storming, for sure.
E.g. whether to go for a "semi-final" API at the first short or to take
a two-stage approach. A legacy API consisting just of the cmd functions
with a new unrelated API in parallel.


Arnim

Any preferences as to a varargs macro, varargs function w/out file/line,
or global variable?
For the varargs macro, I have checked the 3 compilers that I have easy
access to:
- gcc
- MSVC
- SUNWspro cc
All three are fine w/ varargs macros, although SUNWspro cc needs a flag
  -xCC.

As I said, my personal preference currently leans towards a varags 
macro, although I am equally fine w/ the solution that involves a global 
error struct variable.

Rutger

Rutger Hofman wrote:
> For now, discussion of error handling
> =====================================
> 
>>> How should the API functions handle errors? In current cmd, an error
>> We have that common "chain" structure. An integer and string for
>> reporting errors could be placed in there, or even a linked list.
> 
[snip problem explanation]

> So, choices would now seem to be:
> a) use a varargs macro
>     solves all problems but might not be available
> b) use a varags function
>     we cannot get at __FILE__ __FUNCTION__ __LINE__
> c) use a global/glocal struct variable for error handling, not a part of 
> the chain struct. Then we can use the parentheses trick again, because 
> the printer routine knows about the global/glocal struct.
> 
> Note: if we choose a) and configure finds out we don't have varargs 
> macros, the implementation can use b), a varargs function -- with loss 
> of functionality.
> 
> Any preferences? As I said before, we do already depend on c99 features 
> like '//' comments. That attracts me to solution a) with b) as fallback.
> 
> If the list is interested, I can post a sample implementation/invocation 
> for a) so we can take a good close look.
> 
> Rutger
> 
> 
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensign option that enables unlimited
> royalty-free distribution of the report engine for externally facing 
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> UrJTAG-development mailing list
> UrJTAG-development@...
> https://lists.sourceforge.net/lists/listinfo/urjtag-development

Rutger Hofman schrieb:
> Any preferences as to a varargs macro, varargs function w/out file/line,
> or global variable?
> For the varargs macro, I have checked the 3 compilers that I have easy
> access to:
> - gcc
> - MSVC
> - SUNWspro cc
> All three are fine w/ varargs macros, although SUNWspro cc needs a flag
>   -xCC.
> 
> As I said, my personal preference currently leans towards a varags 
> macro, although I am equally fine w/ the solution that involves a global 
> error struct variable.

I'm pretty unbiased on this topic and would follow your proposal either
way.

Arnim

Rutger Hofman wrote:
>> For now, discussion of error handling
>> =====================================

Continued:

How would errors be handled ideally from cmd/cmd_<handler>() and 
urj_cmd_run() ? The cmd handlers call library functions. On error, the 
library routines set error state and return -1/NULL. Now we have two 
possibilities to handle a library error:

a) the cmd handler just returns -1, and urj_cmd_run() prints the error 
description, resets the error, and continues; on command syntax error, 
the handlers set urj_error to URJ_ERROR_SYNTAX or something, and the 
error is handled the same way by urj_cmd_run(); ditto on sanity check 
failure in the handler;

b) the cmd handlers each handle library errors themselves as described 
above for urj_cmd_run() and return 1; they return -1 only if the command 
syntax is wrong

FWIW, I prefer a). This way, the cmd_handler calls are treated no 
differently from other library calls.

Another one: many/all of the cmd handlers get the active part from 
chain, with some sanity checks. Wouldn't we make a function call out of 
this (if only for code sharing)?

Opinions?

Rutger

On Tuesday 21 April 2009 15:04:48 Rutger Hofman wrote:
> Rutger Hofman wrote:
> > Rutger Hofman wrote:
> >> As a preparation for doing the Librarification, I would like to
> >> discuss any changes I will perform in some more detail. More messages
> >> with deeper detail will follow...
>
> What is the value we attach to having the
>
>     /* $Id: <filename> <number> <date> <committer> */
>
> lines? I think this was very handy long ago, but my feeling nowadays is
> that it only generates false hits in svn diff.
>
> Is it OK to remove these lines?

punt them
-mike

On Thursday 23 April 2009 10:13:25 Rutger Hofman wrote:
> Arnim Läuger wrote:
> >>> OK, I will add a vprintf-type routine to chain-> that is called by
> >>> urj_printf(const char *fmt, ...). Should it default to printing on
> >>> stdout or on stderr?
> >>
> >> How about introducing a (simple) "log level"? Maybe
> >
> > We had some thoughts on this earlier, see
> > http://sourceforge.net/tracker/?func=detail&aid=1958007&group_id=193266&a
> >tid=944728
>
> When doing logging/error reporting, it would be good to include file
> name, (function name,) line number in the report. That means that the
> logging function must be a macro (that calls a function with (among
> others) arguments __FILE__, __FUNCTION__, __LINE__).

__func__, not __FUNCTION__.  the former has been standardized.
-mike

On Thursday 23 April 2009 10:17:06 Rutger Hofman wrote:
> Rutger Hofman wrote:
> > (1) make this a varags macro:
> > (1) is a GCC extension.
>
> Mwah, sorry, it's C99. FWIW, we already use C99 features like //-style
> comments.

using C99 features is fine imo.  considering our realistic targets / usage, 
requiring a C99 compiler is sane.
-mike

On Monday 20 April 2009 05:58:31 Rutger Hofman wrote:
> Dick Hollenbeck wrote:
> > 1) Use uncrustify, not indent for the beautification.  I did a detailed
> > search for beautifiers 2 years ago.  Uncrustify wins.   Grab the
> > uncrustify.cfg file from Kicad's repo.  I wrote it myself, it will work
> > on C or C++.   Uncrustify's author will respond to bug reports, and is a
> > fine guy to lean on for this kind of tool.
>
> The current function call/definition style nearly everywhere in UrJTAG
> now has:
>
> int fun( int arg )
> {
> }
>
> x = fun( a );
>
> but the GNU style is:
>
> int fun (int arg)
> {
> }
>
> x = fun (a);
>
> Do we actually want to change to GNU style here? FWIW, I am neutral (I
> prefer: x = fun(a);), although I think GNU style unappealing in:
>
>      printf (_ ("Fatal error"));
>
> If we prefer current practice to GNU style: I seem unable to find an
> indent option for this. OTOH, uncrustify supports it.
>
> One thing that didn't make me love uncrustify immediately: I did apt-get
> install uncrustify on my Ubuntu Hardy system, and it dumps core during a
> call uncrustify --show-config. apt-get uncrustify is 0.41; bleeding edge
> is 0.52, and it doesn't dump.

maybe i spend too much time in lower projects, but the GNU style is awful 
(especially the retarded mixing of spaces and tabs for indentation).

the linux style is what i prefer:
int func(int arg)
{
}
x = func(a);

although i can live with spaces around (...)
-mike

On Saturday 18 April 2009 14:26:15 Dick Hollenbeck wrote:
> 2) Switch to CMake *as you do* the retransformation, not after.  Skip
> libtool, it is obsoleted by CMake as far as I can tell.  And run away
> from autotools immediately.

"obsolete" is wrong.  cmake vs autotools are different ways to approach a 
problem, neither of which obsoletes the other.  autotools work just fine for 
portability and cross-compiling and in my experience, autotools is 
significantly more mature in this area than cmake.

then again, the two arent necessarily exclusive.  the cmake build files can be 
easily overlaid an autotooled package.  so if someone sufficiently motivated 
(i.e. you) cares enough to maintain the cmake files and keep them working, 
then go for it.

> With CMake you can build from the ground up, a few source files at a time.

it isnt that hard: `make foo.o`

> You could build a static libarary during the transformation, and then at the
> end switch to a dynamic target.

again, trivial: --disable-static

> 3) Seriously consider switching to C++ soon, perhaps during the
> transformation.   It is reasonably easy to get C code to compile with
> C++, and that gives you the option of adding exception support later.
> It is not impossible to have code that compiles with either C or C++,
> but that need not be a goal.  The vision I sense here from you guys is
> that you want bindings to other languages, and that suggests a minimum
> host platform, so C++ can be expected there.

unless there are properly defined use cases here where C++ is a clear win, 
then switching to it is much more detrimental.  C++ is a hindrance when 
dealing with embedded systems as you arent merely talking about language 
style.  the C++ runtime is significantly harder to get right and fully 
supported.
-mike

Mike Frysinger wrote:
...
>> logging function must be a macro (that calls a function with (among
>> others) arguments __FILE__, __FUNCTION__, __LINE__).
> 
> __func__, not __FUNCTION__.  the former has been standardized.
> -mike

OK, I'll use that. Some compilers know __func__, some __FUNCTION__, some 
both. autoconf might help here.

Rutger

On Tuesday 28 April 2009 13:36:20 Rutger Hofman wrote:
> Mike Frysinger wrote:
> >> logging function must be a macro (that calls a function with (among
> >> others) arguments __FILE__, __FUNCTION__, __LINE__).
> >
> > __func__, not __FUNCTION__.  the former has been standardized.
>
> OK, I'll use that. Some compilers know __func__, some __FUNCTION__, some
> both. autoconf might help here.

i used "standardized" for a reason: all C99 conforming compilers support it.
-mike

Arnim Laeuger wrote:
> Rutger Hofman schrieb:
>> Any preferences as to a varargs macro, varargs function w/out file/line,
>> or global variable?
>> As I said, my personal preference currently leans towards a varags 
>> macro, although I am equally fine w/ the solution that involves a global 
>> error struct variable.
> 
> I'm pretty unbiased on this topic and would follow your proposal either
> way.

Introduce urj_types.h
=====================

OK, I'll do a varargs macro then. The same for logging stuff.

This will bring one large change: any library function that is capable 
of setting an error must have access to urj_chain_t *chain. Where that 
isn't yet the case, I will fold in an argument urj_chain_t *chain. 
Nearly all prototypes will come to depend on typedef struct urj_chain 
urj_chain_t. This makes the include tree a bit messy because all include 
files must first include "chain.h", which imports lots of other files 
besides the typedef of urj_chain_t. It's a hassle to avoid include loops.

I would like to circumvent this by introducing the ancient and 
time-honoured tric of a file urj_types.h. It would contain *only opaque 
definitions*, starting with typedef struct urj_chain urj_chain_t, and 
possibly with other opaque typedefs for stuff like urj_bus_t. The actual 
struct definition would still be in chain.h or bus.h.

Objections?

Rutger

1. error implementation?
2. global state in one envelope struct?

Rutger Hofman wrote:
> Arnim Laeuger wrote:
>> Rutger Hofman schrieb:
>>> Any preferences as to a varargs macro, varargs function w/out file/line,
>>> or global variable?
>>> As I said, my personal preference currently leans towards a varags 
>>> macro, although I am equally fine w/ the solution that involves a global 
>>> error struct variable.
>> I'm pretty unbiased on this topic and would follow your proposal either
>> way.
[snip]
> OK, I'll do a varargs macro then. The same for logging stuff.

I started doing this but seeing the partial result, I decided to back 
off from this decision. Many, many functions in many modules would 
acquire an urj_cable_t * argument only because they need to set an 
error, and this makes the code look significantly more cluttered. I now 
prefer a global error struct variable. I am also convinced by Mike F's 
arguments concerning c99 :-) so the varargs macro will stay in (unless 
the list objects; it will be straightforward to switch away from).

Result: global variables for error state, logging state, bus, buses, 
cables, endianness, debug mode, ... Wouldn't we want to have one 
envelope struct variable that contains all other global variables?

Rutger

I'll answer to myself, then:

Rutger Hofman wrote:
> Rutger Hofman wrote:
>>> For now, discussion of error handling
>>> =====================================
> 
> Continued:
> 
> How would errors be handled ideally from cmd/cmd_<handler>() and 
> urj_cmd_run() ? The cmd handlers call library functions. On error, the 
> library routines set error state and return -1/NULL. Now we have two 
> possibilities to handle a library error:
> 
> a) the cmd handler just returns -1, and urj_cmd_run() prints the error 
> description, resets the error, and continues; on command syntax error, 
> the handlers set urj_error to URJ_ERROR_SYNTAX or something, and the 
> error is handled the same way by urj_cmd_run(); ditto on sanity check 
> failure in the handler;
> 
> b) the cmd handlers each handle library errors themselves as described 
> above for urj_cmd_run() and return 1; they return -1 only if the command 
> syntax is wrong
> 
> FWIW, I prefer a). This way, the cmd_handler calls are treated no 
> differently from other library calls.

Well, since the command interpreter is also invoked from the library, 
and since the library is supposed to be silent (unless logging is 
enabled), I'll do a).

> Another one: many/all of the cmd handlers get the active part from 
> chain, with some sanity checks. Wouldn't we make a function call out of 
> this (if only for code sharing)?

I'll do this unless somebody objects.

Rutger

One of the possibilities of using UrJTAG as a library must IMHO be to
specify no output from the library at all. That is standard library
practice, after all, and there might be embedded systems that don't have
stdout/stderr at all.

This raises the issue of error handling. There is one correct solution
IMHO: on failure, a library routine pushes error info on the error stack
and returns an error indicator value (like -1 for ints, NULL for
pointers). It is up to the application to handle the error in the way it
prefers.

Now, the error handling in the code is a mixed bag. Sometimes 1 is
returned for failure, sometimes -1, sometimes any negative value, and
there are library routines that have void return type and only do a
printf(error); return;. Turning this into something more uniform is IMHO:
  = necessary
  = laborious
  = error-prone (more so than the identifier alpha conversions)

Caveats: the time I can spend on refactoring UrJTAG is limited, and I
have only one platform to test: a BlackFin BF548 EZ-Kit with a USB gnICE
cable.

About integration with logging: I think this operation in complete
silence must be one of the logging options, so we would have logging
levels like e.g.

0 all
1 comm
2 debug
3 detail
4 normal
5 warnings
6 errors
7 silent

If logging level is not 'silent' and an error is raised, should the
error be printed immediately, besides being thrown to higher levels? I'd
say, yes.

Rutger

Rutger Hofman wrote:
> One of the possibilities of using UrJTAG as a library must IMHO be to
> specify no output from the library at all. That is standard library
> practice, after all, and there might be embedded systems that don't have
> stdout/stderr at all.
>
> This raises the issue of error handling. There is one correct solution
> IMHO: on failure, a library routine pushes error info on the error stack
>   

Rutger,

I am not familiar with this code yet, but I am curious.  What is the 
error stack?  Is this a global data structure?  And is it like a lifo stack?

Thanks,

Dick

Dick Hollenbeck wrote:
> Rutger Hofman wrote:
>> This raises the issue of error handling. There is one correct solution
>> IMHO: on failure, a library routine pushes error info on the error stack
> 
> I am not familiar with this code yet, but I am curious.  What is the 
> error stack?  Is this a global data structure?  And is it like a lifo stack?

It is a global struct. Current plans are to make it max. 1 deep, but we 
would like to design the interface so that a deeper stack is possible. 
openSSL uses an error stack of nontrivial depth.

Rutger

Dick Hollenbeck wrote:
> Rutger Hofman wrote:
>   
>> One of the possibilities of using UrJTAG as a library must IMHO be to
>> specify no output from the library at all. That is standard library
>> practice, after all, and there might be embedded systems that don't have
>> stdout/stderr at all.
>>
>> This raises the issue of error handling. There is one correct solution
>> IMHO: on failure, a library routine pushes error info on the error stack
>>   
>>     
>
> Rutger,
>
> I am not familiar with this code yet, but I am curious.  What is the 
> error stack?  Is this a global data structure?  And is it like a lifo stack?
>
> Thanks,
>
> Dick
>   

Thanks Rutger.

What languages will this library be callable from, and how?

SWIG, or something else?

Dick

On Monday 04 May 2009 05:54:16 Rutger Hofman wrote:
> One of the possibilities of using UrJTAG as a library must IMHO be to
> specify no output from the library at all. That is standard library
> practice, after all, and there might be embedded systems that don't have
> stdout/stderr at all.
>
> This raises the issue of error handling. There is one correct solution
> IMHO: on failure, a library routine pushes error info on the error stack
> and returns an error indicator value (like -1 for ints, NULL for
> pointers). It is up to the application to handle the error in the way it
> prefers.

mmm an "error stack" is pretty uncommon in my experience with libraries.  it 
also makes things inherently difficult in the multithreaded situation.  why 
would an errno-type of variable not be sufficient ?
-mike

Mike Frysinger wrote:
> On Monday 04 May 2009 05:54:16 Rutger Hofman wrote:
>   
>> One of the possibilities of using UrJTAG as a library must IMHO be to
>> specify no output from the library at all. That is standard library
>> practice, after all, and there might be embedded systems that don't have
>> stdout/stderr at all.
>>
>> This raises the issue of error handling. There is one correct solution
>> IMHO: on failure, a library routine pushes error info on the error stack
>> and returns an error indicator value (like -1 for ints, NULL for
>> pointers). It is up to the application to handle the error in the way it
>> prefers.
>>     
>
> mmm an "error stack" is pretty uncommon in my experience with libraries.  it 
> also makes things inherently difficult in the multithreaded situation.  why 
> would an errno-type of variable not be sufficient ?
> -mike
>   

Mike & Rutger,

Is the system multi-threaded?   Have you ever written any JNI type 
code?  This is the C/C++ interface that allows Java code to call C/C++.  
Actually an entire JVM also, has this thingy they call an "execution 
context".  It is the first argument to all the library functions.  It 
gives you your thread safety, and it it extensible because it is 
actually a pointer to an opaque structure.  The nice thing is that you 
can put stuff in there, such as exception text, errore codes, and stacks 
if you want.  I wrote a whole JVM once back in 1995.

Its cost is that you have pass this argument around to all the 
functions, and they in turn pass it amongst themselves.  But it is worth 
the cost I think.  Please don't assume I am recommending this for the 
benefit of Java. 

This is for the benefit of the entire library, regardless of which 
language sits on top.  Some environments have thread specific data, but 
this is a structure pointer that we see being passed to each C function 
withing the library.  It gives you way to support exceptions from within 
C code.  Google for JNIEnv.

Just a thought for the future.

Dick

On Monday 04 May 2009 23:51:15 Dick Hollenbeck wrote:
> Mike Frysinger wrote:
> > On Monday 04 May 2009 05:54:16 Rutger Hofman wrote:
> >> One of the possibilities of using UrJTAG as a library must IMHO be to
> >> specify no output from the library at all. That is standard library
> >> practice, after all, and there might be embedded systems that don't have
> >> stdout/stderr at all.
> >>
> >> This raises the issue of error handling. There is one correct solution
> >> IMHO: on failure, a library routine pushes error info on the error stack
> >> and returns an error indicator value (like -1 for ints, NULL for
> >> pointers). It is up to the application to handle the error in the way it
> >> prefers.
> >
> > mmm an "error stack" is pretty uncommon in my experience with libraries. 
> > it also makes things inherently difficult in the multithreaded situation.
> >  why would an errno-type of variable not be sufficient ?
> > -mike
>
> Mike & Rutger,
>
> Is the system multi-threaded?   Have you ever written any JNI type
> code?  This is the C/C++ interface that allows Java code to call C/C++.
> Actually an entire JVM also, has this thingy they call an "execution
> context".  It is the first argument to all the library functions.  It
> gives you your thread safety, and it it extensible because it is
> actually a pointer to an opaque structure.  The nice thing is that you
> can put stuff in there, such as exception text, errore codes, and stacks
> if you want.  I wrote a whole JVM once back in 1995.
>
> Its cost is that you have pass this argument around to all the
> functions, and they in turn pass it amongst themselves.  But it is worth
> the cost I think.  Please don't assume I am recommending this for the
> benefit of Java.
>
> This is for the benefit of the entire library, regardless of which
> language sits on top.  Some environments have thread specific data, but
> this is a structure pointer that we see being passed to each C function
> withing the library.  It gives you way to support exceptions from within
> C code.  Google for JNIEnv.

C99 already provides a trivial way to do thread local storage.  i dont see any 
need for passing around context structures.

my point was about actually needing a stack of errors vs only having the last 
error available.  the latter is how every library ive seen work and it hasnt 
been a real limitation.
-mike

Mike Frysinger wrote:
> On Monday 04 May 2009 23:51:15 Dick Hollenbeck wrote:
>> Mike Frysinger wrote:
>>> On Monday 04 May 2009 05:54:16 Rutger Hofman wrote:
>>>> One of the possibilities of using UrJTAG as a library must IMHO be to
...
>>>> This raises the issue of error handling. There is one correct solution
>>>> IMHO: on failure, a library routine pushes error info on the error stack
>>>> and returns an error indicator value (like -1 for ints, NULL for
>>>> pointers). It is up to the application to handle the error in the way it
>>>> prefers.
>>> mmm an "error stack" is pretty uncommon in my experience with libraries. 
>>> it also makes things inherently difficult in the multithreaded situation.
>>>  why would an errno-type of variable not be sufficient ?
>>> -mike
>> Mike & Rutger,
>>
>> Is the system multi-threaded?   Have you ever written any JNI type
>> code?

FWIW, I wrote a JNI once (for manta, http://www.cs.vu.nl/manta/; you can 
see that its most recent news dates from 2001).

> C99 already provides a trivial way to do thread local storage.  i dont see any 
> need for passing around context structures.

UrJTAG already has some global data structures: urj_bus, urj_buses, and 
then some (and a number of init-only things like bus/cable driver 
structs). This makes me feel that it is not thread-safe. If liburjtag is 
used by a threaded application, my current guess is that the application 
is responsible for protecting all its UrJTAG accesses.

FWIW, this is fine with me.

> my point was about actually needing a stack of errors vs only having the last 
> error available.  the latter is how every library ive seen work and it hasnt 
> been a real limitation.

Sure, I agree. It is just that somebody mentioned an error stack; I am 
completely content with just an error state (AKA stack of depth 1). For 
the record, OpenSSL uses an error stack, but that is a *huge* library, 
and I can understand that they want to allow access/control to errors in 
the many layers it sports.

Rutger

On Tuesday 05 May 2009 07:25:03 Rutger Hofman wrote:
> Mike Frysinger wrote:
> > C99 already provides a trivial way to do thread local storage.  i dont
> > see any need for passing around context structures.
>
> UrJTAG already has some global data structures: urj_bus, urj_buses, and
> then some (and a number of init-only things like bus/cable driver
> structs). This makes me feel that it is not thread-safe. If liburjtag is
> used by a threaded application, my current guess is that the application
> is responsible for protecting all its UrJTAG accesses.
>
> FWIW, this is fine with me.

i thought you were in the process of pushing those down.  if not, no big deal 
as it isnt a regression from today.

> > my point was about actually needing a stack of errors vs only having the
> > last error available.  the latter is how every library ive seen work and
> > it hasnt been a real limitation.
>
> Sure, I agree. It is just that somebody mentioned an error stack; I am
> completely content with just an error state (AKA stack of depth 1). For
> the record, OpenSSL uses an error stack, but that is a *huge* library,
> and I can understand that they want to allow access/control to errors in
> the many layers it sports.

i would prefer we stick with state rather than stack.  openssl has module 
layers which we dont need to support: you ask the urjtag library to do 
something direct and that's it.
-mike

Mike Frysinger wrote:
> On Tuesday 05 May 2009 07:25:03 Rutger Hofman wrote:
>> Mike Frysinger wrote:
>>> my point was about actually needing a stack of errors vs only having the
>>> last error available.  the latter is how every library ive seen work and
>>> it hasnt been a real limitation.
>> Sure, I agree.
> 
> i would prefer we stick with state rather than stack.

OK, I will do just error state then.

Rutger

> OK, I will do just error state then.

I also prefer the state.

Regarding passing the globals vs. reference argument in every function call.. I
understand that code becomes too clumsy if every small function needs a
reference to some global struct.  So I agree to put things like last-error
information into global variables.  But the library user may want to control
multiple cables/chains concurrently(*), even if there's only one part attached
(**). Therefore, the chain states have to be in different, non-global structs.

(*) Think of the UrJTAG library being used by a JTAG server that connects to
multiple boards in some manufacturing environment.

(*) For instance, it is possible to embed a "virtual JTAG" interface inside an
Altera FPGA to communicate with logic blocks attached to it. To communicate
with the virtual JTAG, the real JTAG is used. Although we don't have it yet in
UrJTAG, I think it's feasible to implement a "altera_virtual_jtag" cable
utilizing another real cable, and state changes in the real cable must not
interfere with the virtual cable...

Kolja

Hi Rutger,

> Caveats: the time I can spend on refactoring UrJTAG is limited, and I

"BTW"

It's absolutely great that you're spending so MUCH time on UrJTAG, and I'm
quite unconfortable with me being so silent in the process at the moment. At
work we're developing a new distance measuring device; I'm responsible for the
system architecture, OS and signal processing and it takes a lot of my
concentration during the day, so I'm quite exhausted after work.

I then just try at least to read what's going on, and the other developers here
also do so, so rest assured that if you're doing something and no one comments,
it's very probably the right thing.

In any case, it doesn't have to be an all-at-once refactoring if that can't be
done within the available time. For example, it doesn't have to become
thread-safe over night and stuff like output levels (error/warning/...) also
can't be all decided correctly in no time. IMHO, it is most important now to
make the API so that UrJTAG /can/ become thread-safe some time in the future,
without too much conversion effort for the API users.

Kolja

Waschk,Kolja wrote:
> Hi Rutger,
> 
>> Caveats: the time I can spend on refactoring UrJTAG is limited, and I
> 
> "BTW"
> 
> It's absolutely great that you're spending so MUCH time on UrJTAG

Well, I am certainly glad w/ this heads-up!

Rutger

Hi Rutger,

> Now, the error handling in the code is a mixed bag. Sometimes 1 is
> returned for failure, sometimes -1, sometimes any negative value, and
> there are library routines that have void return type and only do a
> printf(error); return;. Turning this into something more uniform is IMHO:
>  = necessary
>  = laborious
>  = error-prone (more so than the identifier alpha conversions)

I started to work on this in the past but didn't bite my way through all
the files. This introduced the URJ_STATUS_* return codes for certain
parts of the code base.
Such instances are often accompanied by a printf telling details about
the error mode. E.g. src/bus/ejtag.c line 315. Does your proposal refer
to such spots as well? I.e. to replace the bulk printf with
urj_error_set and return -1? Or should such functions return
urj_error_t URJ_ERROR_OK?

Once this is clear, I could walk through the code base again and care
for such uniformity.

> Caveats: the time I can spend on refactoring UrJTAG is limited, and I
> have only one platform to test: a BlackFin BF548 EZ-Kit with a USB gnICE
> cable.

Like Kolja, I'm also very happy about the tremendous effort you spent
for UrJTAG lately, thank you very much!

> About integration with logging: I think this operation in complete
> silence must be one of the logging options, so we would have logging
> levels like e.g.
> 
> [...]
> 
> If logging level is not 'silent' and an error is raised, should the
> error be printed immediately, besides being thrown to higher levels? I'd
> say, yes.

This implies that the higher level adjusts the logging level accordingly
before calling functions. As an example, urjtag/ started to bark at me
from src/global/parse.c-urj_parse_file() because it doesn't find
$HOME/.jtag/rc. Its presence is optional, so should jtag_parse_rc()
force the logging level to silent before calling urj_parse_file() and
restore the level later on?


Arnim

Arnim Läuger wrote:
>> Now, the error handling in the code is a mixed bag. Sometimes 1 is
>> returned for failure, sometimes -1, sometimes any negative value, and
>> there are library routines that have void return type and only do a
>> printf(error); return;. Turning this into something more uniform is IMHO:
>>  = necessary
>>  = laborious
>>  = error-prone (more so than the identifier alpha conversions)
> 
> I started to work on this in the past but didn't bite my way through all
> the files. This introduced the URJ_STATUS_* return codes for certain
> parts of the code base.
> Such instances are often accompanied by a printf telling details about
> the error mode. E.g. src/bus/ejtag.c line 315. Does your proposal refer
> to such spots as well? I.e. to replace the bulk printf with
> urj_error_set and return -1? Or should such functions return
> urj_error_t URJ_ERROR_OK?

My preference would be that in all cases where an error condition is 
met, the function would urj_error_set(ERROR, "description") and return 
an appropriate error value. It is then up to the application to handle 
the error (or ignore or abort or ...).

In practice my personal preference is:
  - replace all printf-s in the code with urj_log() or urj_error_set().
    In etjag.c:315, that would mean:
      urj_error_set(URJ_ERROR_NOTFOUND, _("EJCONTROL or EJIMPCODE 
register"));
      return URJ_STATUS_FAIL (or -1);
  - if a status is returned: 0 for OK and -1 for error (or URJ_STATUS_OK 
and URJ_STATUS_FAIL)
  - if a nonnegative number is returned: return the number or -1 (or 
URJ_STATUS_FAIL) for error
  - if a possibly negative number is returned: make it an OUT parameter 
and return 0/-1 (OK/FAIL)
  - if a pointer is returned: NULL for error

Using URJ_STATUS_OK and URJ_STATUS_FAIL as aliases is 100% fine with me; 
the only difference is, it doesn't as explicitly show off the behavior 
for nonnegative returns as having -1.

> Once this is clear, I could walk through the code base again and care
> for such uniformity.

I am currently walking the code base too, as you may have noticed. My 
refactoring currently targets that URJ_STATUS_OK or URJ_STATUS_FAIL is 
returned/checked. For void functions that actually return in case of an 
error, I attempted to change the return type to int as a status indicator.

The code under bus/ is by far the largest in line count. I postponed 
that until the last...

The return value behavior of urj_cmd_run() and the specific command 
run()s is different from elsewhere. It has 1: OK or library error; -1: 
syntax error. We might consider if we want to refactor that to reflect 
the practice as outlined above, or if we are content with current practice.

>> Caveats: the time I can spend on refactoring UrJTAG is limited, and I
>> have only one platform to test: a BlackFin BF548 EZ-Kit with a USB gnICE
>> cable.
> 
> Like Kolja, I'm also very happy about the tremendous effort you spent
> for UrJTAG lately, thank you very much!

Well, happy to do all this. Also, having liburjtag allows us to program 
the FPGA on our machine. That's a Great Thing.

>> About integration with logging: I think this operation in complete
>> silence must be one of the logging options, so we would have logging
>> levels like e.g.
>>
>> [...]
>>
>> If logging level is not 'silent' and an error is raised, should the
>> error be printed immediately, besides being thrown to higher levels? I'd
>> say, yes.
> 
> This implies that the higher level adjusts the logging level accordingly
> before calling functions. As an example, urjtag/ started to bark at me
> from src/global/parse.c-urj_parse_file() because it doesn't find
> $HOME/.jtag/rc. Its presence is optional, so should jtag_parse_rc()
> force the logging level to silent before calling urj_parse_file() and
> restore the level later on?

I also noted this, it certainly is "unpleasant". My intention was that 
one global setting for the whole program run would yield a workable 
solution, without having to change the global logging level.

I can think of two solutions. Either urj_error_set() is always silent, 
and the jtag shell or whatever prints the error report -- this would be 
common practice with libraries. Or we decide that this jtag_parse_rc() 
is unusual in that it can suffer absence of the file to open. Then we 
could think of an ad-hoc solution, like an extra log level parameter to 
urj_parse_file(); or have jtag_parse_rc() do a stat() beforehand, to 
explicitly reflect the optional character of ~/.jtag/rc.

Tomorrow, I will be doing different things (meetings). I will continue 
tomorrow night or Thursday.

Rutger

Rutger Hofman wrote:
> Arnim Läuger wrote:

>> Once this is clear, I could walk through the code base again and care
>> for such uniformity.
> 
> I am currently walking the code base too, as you may have noticed.

To avoid any chance of a collision: right now, I am at work in 
src/flash/. I guess bsdl/ will be next.

Rutger

Rutger Hofman wrote:
> Rutger Hofman wrote:
>> Arnim Läuger wrote:
>>> Once this is clear, I could walk through the code base again and care
>>> for such uniformity.
>> I am currently walking the code base too, as you may have noticed.

I refactored src/flash/. It was much less trivial than I would have 
hoped, a.o. because some flash implementations already attempted to 
return smarter error values.

My problem is that I have no flash to test with. Can somebody please 
check if things flashy still work? (I know testing flash stuff is 
dangerous...)

Rutger

P.S. For the moment, I have stuck -Werror into configure.ac, and added 
some prototypes to .l files that flex should have added. I marked them 
so it is easy to have them be gone, when I am done.

Rutger Hofman wrote:
> Arnim Läuger wrote:
>>> If logging level is not 'silent' and an error is raised, should the
>>> error be printed immediately, besides being thrown to higher levels? I'd
>>> say, yes.
>>
>> This implies that the higher level adjusts the logging level accordingly
>> before calling functions. ...
> 
> I can think of two solutions. Either urj_error_set() is always silent, 
> and the jtag shell or whatever prints the error report -- this would be 
> common practice with libraries. Or we decide that this jtag_parse_rc() 
> is unusual in that it can suffer absence of the file to open. Then we 
> could think of an ad-hoc solution ...

I considered for a while. I think there is only one good solution, and 
that is have urj_error_set() be silent whatever the circumstances. This 
means the command shell must handle/print/reset errors when they are 
thrown by the library. For uniformity, I propose to let urj_cmd_run() 
and siblings follow the 'standard interface': be silent when logging 
level requires, and return URJ_STATUS_OK on success, return 
URJ_STATUS_FAIL and set urj_error on failure. One of the failure types 
would be a syntax error. The appropriate print should be done from the 
shell app.

What is your opinion here?

Rutger

Rutger Hofman wrote:
> Rutger Hofman wrote:
>> Arnim Läuger wrote:
> 
>>> Once this is clear, I could walk through the code base again and care
>>> for such uniformity.
>> I am currently walking the code base too, as you may have noticed.
> 
> To avoid any chance of a collision: right now, I am at work in 
> src/flash/. I guess bsdl/ will be next.

Now that flash/ is done, I will start on tap/ and tap/cable/. bsdl had 
better wait until we can think of an easy way to use the existing 
error/warning stuff there.

Rutger

> thrown by the library. For uniformity, I propose to let urj_cmd_run()
> and siblings follow the 'standard interface': be silent when logging
> level requires, and return URJ_STATUS_OK on success, return
> URJ_STATUS_FAIL and set urj_error on failure. One of the failure types
> would be a syntax error. The appropriate print should be done from the
> shell app.
>
> What is your opinion here?

Hi Rutger,

that's a clean and useful solution I think

Kolja

> I considered for a while. I think there is only one good solution, and
> that is have urj_error_set() be silent whatever the circumstances. This
> means the command shell must handle/print/reset errors when they are
> thrown by the library.

I agree, have the lower level generate the message and let the higher
level decide what to do with it.

> For uniformity, I propose to let urj_cmd_run()
> and siblings follow the 'standard interface': be silent when logging
> level requires, and return URJ_STATUS_OK on success, return
> URJ_STATUS_FAIL and set urj_error on failure. One of the failure types
> would be a syntax error. The appropriate print should be done from the
> shell app.
> 
> What is your opinion here?

Yes, good point.

> Now that flash/ is done, I will start on tap/ and tap/cable/. bsdl had
> better wait until we can think of an easy way to use the existing
> error/warning stuff there.

bsdl (and svf) is somewhat special, at least the parser and scanner
parts. There are detours for transporting error information/indication
from parse/lex to the higher level. I'd dig into this since it's my
mess.

Arnim

Waschk,Kolja wrote:
>> thrown by the library. For uniformity, I propose to let urj_cmd_run()
>> and siblings follow the 'standard interface'
> 
> that's a clean and useful solution I think

OK, I'll do src/cmd, src/global, src/apps/ (and possibly src/tap/cable/) 
tomorrow. It's getting late now; off to bed.

Rutger

>> thrown by the library. For uniformity, I propose to let urj_cmd_run()
>> and siblings follow the 'standard interface': be silent when logging
>> level requires, and return URJ_STATUS_OK on success, return
>> URJ_STATUS_FAIL and set urj_error on failure. One of the failure types
>> would be a syntax error. The appropriate print should be done from the
>> shell app.
>>
>> What is your opinion here?

Arnim Läuger wrote:
> Yes, good point.

Waschk,Kolja wrote:
> that's a clean and useful solution I think

Well, there is one case here that doesn't completely fit: how the 'quit' 
command is handled. Three choices:

solve it in jtag_readline_multiple_commands_support() in parse.c:
1) let urj_cmd_run() return a special value URJ_STATUS_QUIT;
2) set an urj_error URJ_ERROR_QUIT;

or solve it in the cmd_quit_run() handler:
3) invoke cleanup code and exit() from the urj_cmd_quit() handler.
Pro: doesn't break the STATUS_FAIL/STATUS_OK paradigm; con: a quit line 
from an included command file would quit the application; also, it would 
be hard to record the quit command in a history file.

Any preferences?

Rutger

Rutger Hofman wrote:
>>> thrown by the library. For uniformity, I propose to let urj_cmd_run()
>>> and siblings follow the 'standard interface': be silent when logging
>>> level requires, and return URJ_STATUS_OK on success, return
>>> URJ_STATUS_FAIL and set urj_error on failure. One of the failure types
>>> would be a syntax error. The appropriate print should be done from the
>>> shell app.

> Well, there is one case here that doesn't completely fit: how the 'quit' 
> command is handled. Three choices:
> 
> solve it in jtag_readline_multiple_commands_support() in parse.c:
> 1) let urj_cmd_run() return a special value URJ_STATUS_QUIT;
> 2) set an urj_error URJ_ERROR_QUIT;
> 
> or solve it in the cmd_quit_run() handler:
> 3) invoke cleanup code and exit() from the urj_cmd_quit() handler.
> Pro: doesn't break the STATUS_FAIL/STATUS_OK paradigm; con: a quit line 
> from an included command file would quit the application; also, it would 
> be hard to record the quit command in a history file.

I chose to return URJ_STATUS_MUST_QUIT.

I did the refactoring in src/cmd/. Now, like the other library routines, 
all cmd_ implementations return URJ_STATUS_OK, URJ_STATUS_FAIL, or 
URJ_STATUS_MUST_QUIT. They don't print on error. The caller (the 
application or a detect implementation or ...) must take care of that.

Well, as I said, this conversion is more error-prone than the previous 
name-changing stuff. I tried to be impossibly careful by checking the 
source changes many times. I tested all commands that I could, but my 
hardware/brain doesn't provide me with the possibility to test the 
following commands:

salias
initbus
dr
get
test
set
peek
poke
pod
readmem
writemem
all flash commands
svf
bsdl

So, please, if somebody can test these commands... I think a very basic 
test for each command would be enough. The implementations in the 
library have not changed, only the calling through a command (or 
include, svf, bsdl). So, if one test succeeds, that ought to be 'proof' 
that all is well for that command.

Rutger

Rutger Hofman wrote:
>>> thrown by the library. For uniformity, I propose to let urj_cmd_run()
>>> and siblings follow the 'standard interface': be silent when logging
>>> level requires, and return URJ_STATUS_OK on success, return
>>> URJ_STATUS_FAIL and set urj_error on failure. One of the failure types
>>> would be a syntax error. The appropriate print should be done from the
>>> shell app.

* About printing of the error: in vanilla UrJTAG, some commands print 
the offending command line upon error (e.g. bit). I thought (and 
implemented) the following is nice orthogonal behavior: when the shell 
detects an error, it just prints the error location + detail; when e.g. 
a library function that invokes 'include' has an error, it is good to 
also always print the offending line, whatever the command. BTW, would 
it also be nice to print the input file name + line number?

* Right now, the error print is too long/redundant. Example from an 
'include' command:

Error: cmd_signal.c:74 cmd_signal_run() already defined: Signal 
'PORTA_0' already defined; command 'signal PORTA_0'

So, the detail message 'Signal 'PORTA_0' already defined' right now 
repeats the error type, 'already defined'.

I attempted to sanitize the error detail messages already, but that most 
certainly appears to be a work in progress.

* This sanitization will also be in the way of internationalization. 
What should be done about that?

Rutger