Menu

Lookup reference table implementation

2024-07-10
2024-07-14
  • Andrew Jameson

    Andrew Jameson - 2024-07-10

    Lookup reference table implementation … trying to get my head around this one. The data that is required by a fixed lookup table is defined by its content. In conventional basic, it might be declared in a Dim statement and its data / content is filled at runtime from a set of data statements. That infers that the array is empty and at runtime is populated by data during initialization. This means that twice the storage is needed for the same data – one in the code and again for the contents of a dynamic array. I disassembled similar PIC hex code and found the same construct – data is transferred into a pre-declared array or memory block … resulting in data duplication and wasted memory resource and initialization code.
    In days gone by, DEC Digital Basic plus 2, featured a MAP statement – not dissimilar to a Pascal record statement … the same block of reserved memory could have many defined views allocated to allow redefinition of its content. Even better, that allocated block of memory content could be pre-initialized during the link process … so at runtime there was no additional code and no data duplication as the block already contained the data.
    Am I wrong in my observation with PIC lookup tables in that data destined for a lookup table is duplicated by the dynamic initialization when that defined table is created ?
    Is there a way where a compiled hex file can contain a fixed predefined / populated data block that is accessed directly by the code itself thereby avoiding duplication of data and initialization code ?

     

    Last edit: Andrew Jameson 2024-07-10
  • Anobium

    Anobium - 2024-07-11

    Lookup reference table implementation … trying to get my head around this one.

    Hopefully I can help.

    The data that is required by a fixed lookup table is defined by its content. In conventional basic, it might be declared in a Dim statement and its data / content is filled at runtime from a set of data statements. That infers that the array is empty and at runtime is populated by data during initialization.

    This is not correct.

    A fixed lookup table is a set of data ( bytes, words etc ) that is stored in the PROGMEM memory. To use: TABLE tablename... data. data... END TABLE, to read use READTABLE

    There is no DIM in the definition process.

    The set of data is part of the hex and it is therefore not filled at runtime.

    This means that twice the storage is needed for the same data – one in the code and again for the contents of a dynamic array. I disassembled similar PIC hex code and found the same construct – data is transferred into a pre-declared array or memory block … resulting in data duplication and wasted memory resource and initialization code.

    The set of data is stored in PROGMEM and there is only one copy.

    If you want to copy the set of data to an array... you could but that would be a redundant process as reading the TABLE can replace the array.

    In days gone by, DEC Digital Basic plus 2, featured a MAP statement – not dissimilar to a Pascal record statement … the same block of reserved memory could have many defined views allocated to allow redefinition of its content. Even better, that allocated block of memory content could be pre-initialized during the link process … so at runtime there was no additional code and no data duplication as the block already contained the data.
    Am I wrong in my observation with PIC lookup tables in that data destined for a lookup table is duplicated by the dynamic initialization when that defined table is created ?
    Is there a way where a compiled hex file can contain a fixed predefined / populated data block that is accessed directly by the code itself thereby avoiding duplication of data and initialization code ?

    Tables using TABLE - END TABLE is the easy way to handle data.

    Review the comments above and ask again.

    Notes:

    If the data set is less than EEPROM in the chip. Use the TABLE but loading it directly to EEPROM ( this is a switch when you define the table in your source), use READTABLE to read the data.

    Advanced:

    TABLE-END TABLE is constrained by PROGMEM page size on the 16F, therefore, 2048 items.

    You could use EEPROM .. END EEPROM this does not use the TABLE/READTABLE. You would need to use a direct method like PROGREAD. All data is stored in EEPROM. Constrained by size of EEPROM and typically byte values.

    You could use DATA .. END DATA this does not use the TABLE/READTABLE. You would need to use a direct method like PROGREAD. All data is stored in PROGMEM. Constrained by size of unused PROGMEM and typically word values, or a max of 0x3FFF for 16F chips.

    For both of the advanced methods you will need the very latest compiler. You may even need a compiler that is not on general release. This depends on the chip you have selected.

     
  • Anobium

    Anobium - 2024-07-11

    And, I should add explanation of array.

    An array is a special type of variable - one which can store several values at once. It is essentially a list of numbers in which each one can be addressed individually through the use of an "index".

    The numbers can be bytes (the default), longs, integers, or words. The index is a value in brackets immediately after the name of the array. An Array is held in RAM.

    To load an array using element addressing or you can load the the whole array. The two examples below explain:

    Load single element, and more single elements. Array are not initialised.

    Dim myArray(10)
    
    myArray(1) = 1
    myArray(2) = 2
    ..
    myArray(0) = 10
    

    To load multiple elements

    Dim myArray(10)
    myArray = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
    

    Array are easy to use but highly costly in terms of RAM and PROGMEM. Compare these two approaches. In both examples the variable myResult will contain the specific data item.

    Using as array uses more PROGMEM and more RAM., both yield the same result.

    #CHIP 18F2550
    #option Explicit
    
    Dim myResult, myIndex as Byte
    
    // Using an array 64 words Progmem / 13 bytes RAM
        Dim myArray(10)
            myArray = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
        For myIndex = 1 to 10
            myResult =  myArray(myIndex)
        Next
    
    #CHIP 18F2550
    #option Explicit
    
    Dim myResult, myIndex as Byte
    
    // Using a table 56 words Progmem / 2 bytes RAM
         For myIndex = 1 to 10
             ReadTable myTable, myIndex, myResult
         Next
    
    
    Table myTable
        1, 2, 3, 4, 5, 6, 7, 8, 9, 10
    End Table
    

    The rational for ReadTable... provide data set capabilities to chips will limited RAM. But, ReadTable is still great today as it is faster and uses less resources.

    Advanced

    A byte array is handled the same a string. String are known to be resource hungry. :-)

     

    Last edit: Anobium 2024-07-11
  • Andrew Jameson

    Andrew Jameson - 2024-07-11

    Wow, I'm so impressed - it's like reading straight from a textbook !

    It'll take me a while to digest. I'm still struggling a little ... it's comprehending what is written directly to the PIC by the hex flash ... is it just a contiguous block of data ? literally dumping the hex file en masse ?

    My problem is that I like to understand the nuts and bolts ... Basic is OK but then understanding the compiled output and its efficiency is so important too.

    I'm using a Dim statement :

    Dim RPMTable(cRPMTableSize) As Word ; Table of selectable RPM speeds ...
    RPMTable(1) = c500RPM ;
    RPMTable(2) = c1000RPM ;
    RPMTable(3) = c2000RPM ;
    RPMTable(4) = c3000RPM ;
    RPMTable(5) = c4000RPM ;
    and
    TargetRPM = RPMTable(idxSpeed)
    this results in :
    movlw 20
    banksel SYSRPMTABLE_1
    movwf SYSRPMTABLE_1
    movlw 58
    movwf SYSRPMTABLE_1_H
    ;RPMTable(2) = c1000RPM ;
    movlw 200
    movwf SYSRPMTABLE_2
    movlw 28
    movwf SYSRPMTABLE_2_H
    ;RPMTable(3) = c2000RPM ;
    movlw 34
    movwf SYSRPMTABLE_3
    movlw 14
    movwf SYSRPMTABLE_3_H
    ;RPMTable(4) = c3000RPM ;
    movlw 64
    movwf SYSRPMTABLE_4
    movlw 9
    movwf SYSRPMTABLE_4_H
    ;RPMTable(5) = c4000RPM ;
    movlw 207
    movwf SYSRPMTABLE_5
    movlw 6
    movwf SYSRPMTABLE_5_H

    It's the relationship between the assembly output and what is actually written to the PIC ... at the moment it looks like the empty array is filled by initialization code and looks like duplication of the array values.

    I'll rewrite the table using the Table construct and examine what that looks like.

    The salient thing is that it all works and it's just the purist who wants to understand a little more.

    Thanks very much for your input it's a fun learning curve. As you've probably surmised, I'm an ageing programmer - years ago my company provided us with Motorola 6800 evaluation kits, it had a hex keypad and 512 bytes of memory. I managed to write a ping pong game that relied on a couple of a D/A converters, an oscilloscope as the display ... it had two collision angles from the paddles and on screen scoring with raster numerals ... it all fitted with 4 bytes free !

    Andrew

     

    Last edit: Andrew Jameson 2024-07-14
  • Anobium

    Anobium - 2024-07-11

    Looking at your code segment. Are those Constants defined somewhere else in the code?

    The asm shows word variables that represent the array, for the high and low word bytes, for each element. The compiler has optimised the array to word variables as this is the fastest/least code method. So, the asm shows the Constant values being loaded into RAM word variables.

    The compiler is very smart. And, thr ASM generated may be different for another chip family or array loading method.

    Enjoy Tables... they will make more sense to you, with your background.

     
  • Anobium

    Anobium - 2024-07-12

    I turned into a paper.

    I have the Word document. If you want to edit/amend please ask for the master document.

    Evan

     
    ❤️
    1
  • Andrew Jameson

    Andrew Jameson - 2024-07-12

    Extremely good, downloaded and filed !

    Great fun ... and now all makes sense - used Table construct and the ASM code now looks like I wanted without obvious redundancy / duplication ... the table that I'm using is small so moving from Dim to Table incurred 4 more bytes but who's counting !

    I love PIC programming, reminds me so much of the "old" days using PDPs / VAXes not to mention 6502s /6800s ... got hard pressed to write code for an industrialized BBC micro and ended up writing a VAX based 6502 cross-compiler - even managed to get the 6502 to run threaded code by stack swapping. (My background was actually Radio / TV Broadcasting - IBA / CH4 days ... so we had to be fairly adept with our engineering skills.)

    I like it as it all takes greater precision and careful thought and it's very much akin to a challenging puzzle to keep code fast, robust and compact.

    You provide a great piece of work that so many must really enjoy ...

    ... thanks

    Andrew

     
    ❤️
    1

    Last edit: Andrew Jameson 2024-07-12
    • Anobium

      Anobium - 2024-07-12

      Pleasure.

      As you say. Your code technique is reflected in the ASM. The compiler will optimise code but you are bare metal programming with the target microcontroller being one of the 1300+ supported.

      What chip are you using?

       
    • Chris Roper

      Chris Roper - 2024-07-12

      Sounds like a similar background but I skipped the VAX portion and started with 6800 chips and even developed and taught a post grad course based on them. but my favorite Chip was the 6809 and to this day I can still write in Hex Code for that device.

      I never had an Apple PC so skipped the 6502 entirely, but did get an Apple Lisa (pre Mac) and had to learn 68000 assembler. The 68000 code was a great asset later in life when the PC department I ran got merged into the Mainframe department. Determined to show this mainframe rookie how real computers worked they gave me a stack of manuals and told me I have a month to lean how to program the IBM 370 Mainframe.

      Amongst the documentation I found a Machine code Reference card that was almost an identical instruction set to the 68000. A few days later I demonstrated the IBM PC using an IBM Tape Drive as a Backup device for MSDOS.

      They were impressed by the code, especially in the short time it took, but did not like having the Mainframe being treated as a slave to the PC :) I did however get handed total control of system integration between PC's, mainframes and the New fangled PC network concept.

      When I moved back into the Electronics world a few years later I was presented with the PIC and it was love at first sight. After several years of coding the PIC in Assembler I discovered GCBASIC and it made it an even better experience as it was easy to code and it could generate better ASM output that I could in most cases.

      Cheers
      Chris

       
      ❤️
      1

      Last edit: Chris Roper 2024-07-12
      • Andrew Jameson

        Andrew Jameson - 2024-07-13

        During my time at Glasgow University, I worked evening shifts in the DP (Data Processing) department - in those days, loading punched cards into readers. Access to the IBM360 was very limited for students so it gave me chance to run my own work - all Fortran 77 and made a little beer money at the same time.
        Later, I bought a UK101 kit (6502) and subsequently an Apple II ... a big investment in those days, around £1000 and that was in the late 70's !
        It had many obvious limitations, the worst being its lack of lowercase characters. A bit of research enabled me to buy a new compatible character ROM 2513 with lowercase characters. I recollect the disappointment in finding that access to lowercase was blocked by two things - no code for the shift key and a firmware based uppercase character mask associated with accessing the character ROM - never could understand why on earth it was there. At this time I wrote to Steve Wozniak and asked for his comment. I got a reply suggesting that it would need new firmware and hardware modifications and not something that they wanted to do at that time.
        I didn't give up and discovered a user settable jump vector during the DOS boot initialization. I was able to run my own code that modified the 6502 stack and replaced the return address of a pushed address for an RTS that was associated with a keyboard press and another addressing the ROM. So I ended up with lowercase functionality and a quirky shift key that involved pressing the control and escape keys together. There were no hardware modifications but I needed to modify the DOS boot code to include my routines.
        There then ensued frequent correspondence with Steve Wozniak and Steve Jobs too and my efforts were rewarded by free deliveries of Windfall, the Apple house magazine and a new colour monitor.
        It was shortly afterwards that they launched the Apple II Europlus with lowercase support.
        I've always kicked myself for not keeping the correspondence - it's like here, just chat and open ideas without an awareness of future consequences / outcomes.
        I often wonder if we've really made progress since those days - with only 48 kilobytes of RAM and a 1MHz CPU, you had immediate access to word processors and spreadsheets and an almost instant boot ... mind you at today's prices an equivalent Apple II would cost £5,300 !

        Good days !

        Andrew

         
        ❤️
        1

        Last edit: Andrew Jameson 2024-07-13
    • Anobium

      Anobium - 2024-07-13

      https://github.com/GreatCowBASIC/Help/blob/main/source/refererencedata.adoc

      I took the posts, edited and turned into a page for the Help.

       
  • Andrew Jameson

    Andrew Jameson - 2024-07-12

    Started with 12F683 but now programming 16LF18313 ... I wanted low power and lower operating voltage.

    It's all about this work ...
    https://vimeo.com/600358859

    The graphics are mine, Delphi / OpenGL ... I wrote a 3D version of Scrabble - Letter Rack 3D and the opening logo is part of that work. I'm now working on a PIC controlled magnetically levitated motor. The challenge is to get it to auto-start and run 24 / 7 from ambient room lighting and at 1000 RPM !

    The circuit diagram in the video featured a step-up voltage converter driven by the coil's reverse back emf / spike that charged a capacitor - up to 9V, so it was enough to overcome the forward voltage of an LED - however as the motor efficiency improved there was not enough power recovered - so it just makes a blue LED flash and the others stop working when the supply voltage drops below 2V.

    Slowly getting there and average power consumption is now less than 100uA - it's hard to measure, as it's so variable and pulsed, so I use a charged 1F supercapacitor as a reference and that'll run it for almost 2 days.

    The latest software can gauge the rotor frictional losses by periodically measuring its free runtime.

    It's quite a rewarding challenge - you'd think that fine pulse width control would be good but then the PIC's awake too much ... the12F683 was a bit power hungry but with the 16LF18313 it'll run on a 1" x 1.5" solar cell placed on a window sill.

    I want to wrap it all into a challenge for my old schools in Edinburgh and Bolton - I had a physics teacher who challenged us all to devise a "novel" motor by the end of year - only criteria was that it was built from domestic rubbish and that there was something that made it "novel". I won his challenge but I'm still going !

    A teacher in my primary school in Edinburgh introduced me to crystal radios - the concept of "free energy" powering headphones was the key motivator for my career direction.

    Such challenges encourage not only learning but a belief in one's self - the SoftSpot logo in the video was important to me and I always wanted a physical reproduction - my daughter in London took this on as her challenge - it took half of her department in Sky TV and a lot of drawings shuttling between there and China but a year later she gave me a solid stainless steel replica complete with spinning globes as Christmas present !

    Sadly my wife was picked up with cancer late last year so time has become a little more precious and playtime has been curtailed a little.

    Cheers Andrew

     
    ❤️
    1

    Last edit: Andrew Jameson 2024-07-12
    • Anobium

      Anobium - 2024-07-14

      A 370 mam myself. The engineers got the 370 for computational FD at night. We have fun the next morning with the dumps when things did not go as planned.

      My story, like yours, was with Ray Ozzy ( Lotus, IBM and Groove Networks ). Man was a genius in understanding data transmission. We know his great work today as OneDrive. Many of my early day Microsoft collegues are now dead, it happens with time. I fondly remember Kim Cameron working on the Rules of Identity Management.

      I worked for Cargill Inc. and was very fortunate to lead Identity Management Strategy for the company. Meant I met and discussed the nuts and bolts of email, LDAP, PKI with BillG, Kim and Eric Schimt ( at the time at Novell ). I ended up being offering the role of leading a new product team in Redmond... it became the X-Box team.

      Years passed by.. I still wanted to be an Engineer. Found GGCBASIC and Hugh... now I am an Engineer. :-)

       
      ❤️
      1
  • Anobium

    Anobium - 2024-07-13

    Impressive work. Great to see the video.

    My thoughts go to your wife and you. Enjoy every moment you can.

     
  • Fabrice Engel

    Fabrice Engel - 2024-07-14

    Hi Guys, I am just so impressive what you achieved in the past, and now you give all back to the community. My best thanks for that !

     
    ❤️
    1

Log in to post a comment.