backuppc-users — General list for user discussion, questions and support

Re: [BackupPC-users] BackupPC and Prometheus.

[Matthew P. <ma...@co...>]

On Tue, Oct 21, 2025 at 1:42 PM G.W. Haywood <ba...@ju...>
wrote:

> It doesn't look like there will be a lot of work involved, but there
> seem to be conflicts in the suggestions which may need to be resolved.
> I've never met Prometheus, but I do use Icinga extensively so I guess
> it won't be terribly unfamiliar.  Can anyone here help me with this?
>

Prometheus and Incus have very little in common. :)  Prometheus has more in
common with something like MRTG (in that it implements the polling and data
storage components).

Prometheus records data in a time series database, and provides a query
language for retrieving that data in ways that make it easily graphable and
easy to generate alerts from.   It's commonly paired with Graphite and
various other tools that feed into push notification (alerting) software.

Prometheus operates by having an "exporter" run on remote systems that
provides an HTTP(s) endpoint that dumps current statistics counters
(key/value).  Exporters are not dissimilar to SNMPD, just with a different
interface and security profile.   Prometheus scrapes the data from whatever
exporters it's configured to access, at regular intervals, and stores the
retrieved statistics in its time series database to be looked at later by
some other process (e.g. Graphite).

Some software comes with an exporter built in (for full circularity,
Graphite provides a Prometheus exporter endpoint, and then provides its own
Dashboard showing its own performance based on the data stored in
Prometheus).  For other software there may be an additional process that
collects and collates data into statistics and provides the endpoint (e.g.,
see the "Node Exporter", which provides system statistics for an OS... this
is really the only place there is overlap with Icinga/Nagios).
BackupPC_Admin?action=metrics?format=prometheus
implements an exporter for Backup PC.


I have only done a very brief review of the issues you mention... It looks
like the conflicts come from different people trying to solve the same
ongoing issue in different ways, depending on their personal needs.  My
personal opinion is that "fix this problem by fixing the value" is a better
approach than "fix this problem by removing the value."   Beyond that any
conflicts arising in what is considered "fixing" a broken statistic is
probably going to require some more knowledge of internals than I have time
to dig into right now.

[BackupPC-users] BackupPC and Prometheus.

[G.W. H. <ba...@ju...>]

Hi there,

The following BackupPC issues/PRs on Github reference Prometheus:

#444
#517
#530
#531

My comment [*] in #444 seems to have elicited no response.

It doesn't look like there will be a lot of work involved, but there
seem to be conflicts in the suggestions which may need to be resolved.
I've never met Prometheus, but I do use Icinga extensively so I guess
it won't be terribly unfamiliar.  Can anyone here help me with this?

[*][quote]
I'd like to think that I will be able to release a new version of
BackupPC this year, and that it would include a fix for this issue.

If anyone is willing to explain in detail (to someone who still thinks
that Prometheus is a Greek god) what this and the other issues/PRs
mentioned are trying to achieve, please either comment here or drop an
email to the BackupPC users' list.

At present I'm unable to devote the time to it that I'd like, but if
it's just a matter of correcting the calculation/rendering of values,
I'd have the time and be very happy to tweak some Perl code in an
existing module and push that.
[/quote]

-- 

73,
Ged.

[BackupPC-users] A couple of BackupPC Web UI fixes.

[G.W. H. <ba...@ju...>]

Hi there,

One small irritation that I've noticed on three different browsers on
two different desktops is that on BackupPC's Web UI, the BackupPC logo
(at the top left hand corner of the page) doesn't fit in the allocated
space.  While fixing Github issue #509 I fixed this minor annoyance in
the CSS too.  My wife provided the inspiration, I'm no CSS expert. :)

These fixes are pretty straightforward.  I tested on Chromium, Firefox
and Pale Moon where all seems OK but I'd be grateful for any feedback.

It's all in master if anyone would like to give it a whirl.

-- 

73,
Ged.

Re: [BackupPC-users] BackupPC_backupDelete deletes entire backup instead of part of a share.

[G.W. H. <ba...@ju...>]

Hi there,

On Sat, 11 Oct 2025, bac...@ko... wrote:

> ... it would be nice if the function would be allow you to
> alternatively specify just a path without a share name similar to
> how directories are stored in the 'pc' directory or how they are
> mounted when using backuppcfs.  i.e., if your share is named 'root'
> (for '/') then you could just do: BackupPC_backupDelete -h HOSTNAME
> -n 742 -s /root/etc/something ...

Do people have sharenames which contain the '/' character?  See e.g.
https://sourceforge.net/p/backuppc/mailman/message/22630073/.  Even if
they do, this could (I think) still be just about doable.  Not part of
this fix though, it would probably take some time to work through.

> ... If your only share is actually '/' then you could do:
> BackupPC_backupDelete -h HOSTNAME -n 742 /etc/something

If I understand and if 'something' is a directory, seems easy enough:

Count the shares; if there's only one, then take the command

BackupPC_backupDelete -h HOSTNAME -n 742 /etc/something

to mean what would right now be given as

BackupPC_backupDelete -h HOSTNAME -n 742 -s OnlyExistingShare /etc/something

and proceed on that basis; otherwise call it a syntax error.

What's the largest number of shares there are likely to be?  One might
even consider, if a path is given without a sharename, looping through
all existing shares.

> It may also be nice to have the option to confirm before deleting...

Noted.  Then we'd have to make sure BackupPC_dump didn't use it... :)

-- 

73,
Ged.

Re: [BackupPC-users] BackupPC_backupDelete deletes entire backup instead of part of a share.

[<bac...@ko...>]

Second patch seems better.
Ideally, though it would be nice if the function would be allow you to
alternatively specify just a path without a share name similar to how
directories are stored in the 'pc' directory or how they are mounted
when using backuppcfs.
i.e., if your share is named 'root' (for '/') then you could just do:
BackupPC_backupDelete -h HOSTNAME -n 742 -s /root/etc/something
If your only share is actually '/' then you could do:
BackupPC_backupDelete -h HOSTNAME -n 742 /etc/something

It may also be nice to have the option to confirm before
deleting... (like with 'rm -i')

G.W. Haywood wrote at about 17:42:49 +0100 on Friday, October 10, 2025:
 > Hi there,
 > 
 > The oldish Github issue
 > 
 > https://github.com/backuppc/backuppc/issues/519
 > 
 > describes the unfortunate situation in the subject.
 > 
 > The trigger was a syntax error when using the BackupPC_backupDelete
 > script from the command line:
 > 
 > # Do not do this!  At least not until there's a fix for #519.
 > $ BackupPC_backupDelete -h HOSTNAME -n 742 -s /etc/something
 > 
 > In this case the name of the share is missing from the command.
 > 
 > BackupPC_backupDelete issues seem to have caused little traffic here
 > on the List.  I wondered if anyone has come across anything like this?
 > The most recent mention of anything similar that I found in the List
 > archives was from me, back in September 2018.  So no, I don't use the
 > script from the command line very often.
 > 
 > I've put a couple of patches in the comments for #519.  The first is a
 > simple-minded thing.  However I'd prefer something more robust, which
 > I hope the second patch would prove to be.  But before I'd be happy to
 > release it, I'd like it to have more eyes on it and more exercise.
 > 
 > If anyone would like to help test it I'd be grateful to hear either on
 > the List, over on the Github issue comments, or privately.  There will
 > be enough information in #519 to patch the script, but I'd be happy to
 > mail the updated version or upload it someplace for download.  I'm not
 > keen on branching/forking on Github but it's an option.
 > 
 > My patch should have no effect on BackupPC's use of the script.  I'm
 > pretty sure that this issue can only affect usage of BackupPC_delete
 > at the command-line, but other eyes on the code would be most welcome.
 > 
 > -- 
 > 
 > 73,
 > Ged.
 > 
 > 
 > _______________________________________________
 > BackupPC-users mailing list
 > Bac...@li...
 > List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
 > Wiki:    https://github.com/backuppc/backuppc/wiki
 > Project: https://backuppc.github.io/backuppc/

[BackupPC-users] BackupPC_backupDelete deletes entire backup instead of part of a share.

[G.W. H. <ba...@ju...>]

Hi there,

The oldish Github issue

https://github.com/backuppc/backuppc/issues/519

describes the unfortunate situation in the subject.

The trigger was a syntax error when using the BackupPC_backupDelete
script from the command line:

# Do not do this!  At least not until there's a fix for #519.
$ BackupPC_backupDelete -h HOSTNAME -n 742 -s /etc/something

In this case the name of the share is missing from the command.

BackupPC_backupDelete issues seem to have caused little traffic here
on the List.  I wondered if anyone has come across anything like this?
The most recent mention of anything similar that I found in the List
archives was from me, back in September 2018.  So no, I don't use the
script from the command line very often.

I've put a couple of patches in the comments for #519.  The first is a
simple-minded thing.  However I'd prefer something more robust, which
I hope the second patch would prove to be.  But before I'd be happy to
release it, I'd like it to have more eyes on it and more exercise.

If anyone would like to help test it I'd be grateful to hear either on
the List, over on the Github issue comments, or privately.  There will
be enough information in #519 to patch the script, but I'd be happy to
mail the updated version or upload it someplace for download.  I'm not
keen on branching/forking on Github but it's an option.

My patch should have no effect on BackupPC's use of the script.  I'm
pretty sure that this issue can only affect usage of BackupPC_delete
at the command-line, but other eyes on the code would be most welcome.

-- 

73,
Ged.

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Hi!

> Did it shed any light on the cause?

Absolutely none 😊

-----Original Message-----
From: G.W. Haywood <ba...@ju...>
Sent: 08 October 2025 13:58
To: bac...@li...
Subject: Re: [BackupPC-users] Missing pool files

Hi there,

On Wed, 8 Oct 2025, Jamie Burchell wrote:

> Found it with:
> ...
> find . -name "poolCnt.1.a0" -exec sh -c ...

Neat. :)

Did it shed any light on the cause?

-- 

73,
Ged.


_______________________________________________
BackupPC-users mailing list
Bac...@li...
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Missing pool files

[G.W. H. <ba...@ju...>]

Hi there,

On Wed, 8 Oct 2025, Jamie Burchell wrote:

> Found it with:
> ...
> find . -name "poolCnt.1.a0" -exec sh -c ...

Neat. :)

Did it shed any light on the cause?

-- 

73,
Ged.

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Well, removing the backup has somehow now revealed the file in question:

2025-10-07 18:30:10 BackupPC_backupDelete: removing #1437
2025-10-07 18:30:10 BackupPC_backupDelete: Merge into backup 1436
2025-10-07 18:30:10 Can't read attribute file
/var/lib/BackupPC//pc/myhost/1436/f%2fvar%2fwww/freleases/f220/fweb/fapp/flanguages/fplugins/attrib_a06340ddb577fd02a04bda2998a8d246
2025-10-07 18:30:20 bpc_attrib_dirRead: can't open
/var/lib/BackupPC//cpool/a0/62/a06340ddb577fd02a04bda2998a8d246
2025-10-07 18:30:22 bpc_attrib_dirRead: can't open
/var/lib/BackupPC//cpool/a0/62/a06340ddb577fd02a04bda2998a8d246
2025-10-07 18:30:22 bpc_path_refCountAll: can't read attrib file
/var/lib/BackupPC//pc/myhost/1436/f%2fvar%2fwww/freleases/f220/fweb/fapp/flanguages/fplugins/attrib_a06340ddb577fd02a04bda2998a8d246
2025-10-07 18:30:28 BackupPC_refCountUpdate: host dmtt_web got 1 errors
(took 7 secs)
2025-10-07 18:30:28 BackupPC_refCountUpdate total errors: 1
2025-10-07 18:30:28 BackupPC_backupDelete: got 2 errors


-----Original Message-----
From: Jamie Burchell <ja...@ib...>
Sent: 07 October 2025 18:29
To: General list for user discussion, questions and support
<bac...@li...>
Subject: RE: [BackupPC-users] Missing pool files

> Found it with:
> # a0 is the first two letters of the hash of the file we are looking for:
> a06340ddb577fd02a04bda2998a8d246
> $ find . -name "poolCnt.1.a0" -exec sh -c 'for f; do
> /usr/share/BackupPC/bin/BackupPC_poolCntPrint "$f" | grep -q
> "a06340ddb577fd02a04bda2998a8d246" && echo "Match: $f"; done' sh {} +

Well having apparently found the backup that this file relates to:

sudo -ubackuppc ./BackupPC_ls -R -h myhost -n 1437 -s /var/www / | egrep
a06340ddb577fd02a04bda2998a8d246

... returns no results. No results in any share of that backup.

-----Original Message-----
From: Jamie Burchell <ja...@ib...>
Sent: 07 October 2025 17:26
To: General list for user discussion, questions and support
<bac...@li...>
Subject: RE: [BackupPC-users] Missing pool files

Found it with:

# a0 is the first two letters of the hash of the file we are looking for:
a06340ddb577fd02a04bda2998a8d246
find . -name "poolCnt.1.a0" -exec sh -c 'for f; do
/usr/share/BackupPC/bin/BackupPC_poolCntPrint "$f" | grep -q
"a06340ddb577fd02a04bda2998a8d246" && echo "Match: $f"; done' sh {} +

Thanks for the pointers!


--


-----Original Message-----
From: G.W. Haywood <ba...@ju...>
Sent: 07 October 2025 14:58
To: bac...@li...
Subject: Re: [BackupPC-users] Missing pool files

Hi there,

On Tue, 7 Oct 2025, G.W. Haywood wrote:
> On Tue, 7 Oct 2025, Jamie Burchell wrote:
>
> Out of the blue on the 24th September, my log showed:
> ...
>
> > [snip]

Oh, I almost forgot:

https://github.com/backuppc/backuppc/wiki/How-to-find-which-backups-reference-a-particular-pool-file

:)

-- 

73,
Ged.


_______________________________________________
BackupPC-users mailing list
Bac...@li...
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

> Found it with:
> # a0 is the first two letters of the hash of the file we are looking for:
> a06340ddb577fd02a04bda2998a8d246
> $ find . -name "poolCnt.1.a0" -exec sh -c 'for f; do
> /usr/share/BackupPC/bin/BackupPC_poolCntPrint "$f" | grep -q
> "a06340ddb577fd02a04bda2998a8d246" && echo "Match: $f"; done' sh {} +

Well having apparently found the backup that this file relates to:

sudo -ubackuppc ./BackupPC_ls -R -h myhost -n 1437 -s /var/www / | egrep
a06340ddb577fd02a04bda2998a8d246

... returns no results. No results in any share of that backup.

-----Original Message-----
From: Jamie Burchell <ja...@ib...>
Sent: 07 October 2025 17:26
To: General list for user discussion, questions and support
<bac...@li...>
Subject: RE: [BackupPC-users] Missing pool files

Found it with:

# a0 is the first two letters of the hash of the file we are looking for:
a06340ddb577fd02a04bda2998a8d246
find . -name "poolCnt.1.a0" -exec sh -c 'for f; do
/usr/share/BackupPC/bin/BackupPC_poolCntPrint "$f" | grep -q
"a06340ddb577fd02a04bda2998a8d246" && echo "Match: $f"; done' sh {} +

Thanks for the pointers!


--


-----Original Message-----
From: G.W. Haywood <ba...@ju...>
Sent: 07 October 2025 14:58
To: bac...@li...
Subject: Re: [BackupPC-users] Missing pool files

Hi there,

On Tue, 7 Oct 2025, G.W. Haywood wrote:
> On Tue, 7 Oct 2025, Jamie Burchell wrote:
>
> Out of the blue on the 24th September, my log showed:
> ...
>
> > [snip]

Oh, I almost forgot:

https://github.com/backuppc/backuppc/wiki/How-to-find-which-backups-reference-a-particular-pool-file

:)

-- 

73,
Ged.


_______________________________________________
BackupPC-users mailing list
Bac...@li...
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Found it with:

# a0 is the first two letters of the hash of the file we are looking for:
a06340ddb577fd02a04bda2998a8d246
find . -name "poolCnt.1.a0" -exec sh -c 'for f; do
/usr/share/BackupPC/bin/BackupPC_poolCntPrint "$f" | grep -q
"a06340ddb577fd02a04bda2998a8d246" && echo "Match: $f"; done' sh {} +

Thanks for the pointers!


--


-----Original Message-----
From: G.W. Haywood <ba...@ju...>
Sent: 07 October 2025 14:58
To: bac...@li...
Subject: Re: [BackupPC-users] Missing pool files

Hi there,

On Tue, 7 Oct 2025, G.W. Haywood wrote:
> On Tue, 7 Oct 2025, Jamie Burchell wrote:
>
> Out of the blue on the 24th September, my log showed:
> ...
>
> > [snip]

Oh, I almost forgot:

https://github.com/backuppc/backuppc/wiki/How-to-find-which-backups-reference-a-particular-pool-file

:)

-- 

73,
Ged.


_______________________________________________
BackupPC-users mailing list
Bac...@li...
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Thanks Ged, but the pool file is missing, so I don't think any of this
helps? Certainly there is no file with the name
a06340ddb577fd02a04bda2998a8d246 anywhere in the pool.

-----Original Message-----
From: G.W. Haywood <ba...@ju...>
Sent: 07 October 2025 14:58
To: bac...@li...
Subject: Re: [BackupPC-users] Missing pool files

Hi there,

On Tue, 7 Oct 2025, G.W. Haywood wrote:
> On Tue, 7 Oct 2025, Jamie Burchell wrote:
>
> Out of the blue on the 24th September, my log showed:
> ...
>
> > [snip]

Oh, I almost forgot:

https://github.com/backuppc/backuppc/wiki/How-to-find-which-backups-reference-a-particular-pool-file

:)

-- 

73,
Ged.


_______________________________________________
BackupPC-users mailing list
Bac...@li...
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Missing pool files

[G.W. H. <ba...@ju...>]

Hi there,

On Tue, 7 Oct 2025, G.W. Haywood wrote:
> On Tue, 7 Oct 2025, Jamie Burchell wrote:
> 
> Out of the blue on the 24th September, my log showed:
> ...
>
> > [snip]

Oh, I almost forgot:

https://github.com/backuppc/backuppc/wiki/How-to-find-which-backups-reference-a-particular-pool-file

:)

-- 

73,
Ged.

Re: [BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Hi Ged

> Even if there were such files, your ls command wouldn't list them
> because it looks for files named 'poolCnt' not 'poolCnt*'. :(

Yes, apologies I pasted the wrong command in to my email - I did actually
try to find with the wildcard version (unsuccessfully).

> Did a machine experience a power failure, crash, or something like
> that?  Perhaps someone restarted BackupPC at an inconvenient time?
> Did you experiment with compression?  Manually fiddle with the pool?

Certainly not changed any settings or fiddled with the pool in any way. It's
possible that the server (cloud based VM) ran in to an issue but I'm not
aware of it.

> backuppc@server:$ /usr/local/BackupPC/bin/BackupPC_ls -R -h host44 -n
> 2072 -s Config / | head

Hmm, but I would need to manually run this for every host (50 of them) and
every backup?

Thanks
Jamie

Re: [BackupPC-users] Missing pool files

[G.W. H. <ba...@ju...>]

Hi there,

On Tue, 7 Oct 2025, Jamie Burchell wrote:

> Out of the blue on the 24th September, my log showed:
> ...
> ...
> 2025-09-24 01:02:30  admin1 : BackupPC_refCountUpdate: missing pool file
> ee88a6fce8449682d6b4465f89b447ec count 19
>
> 2025-09-24 01:03:26  admin1 : BackupPC_refCountUpdate total errors: 9
> ...
> Logs prior to this show no missing pool files and logs after this show a
> decreasing number of missing files. Only one has remained:
> ...
> a06340ddb577fd02a04bda2998a8d246
> ...
> Curious which backup this pertains to, I found and run the script mentioned
> ...
> No log file is created and I cannot find a file named poolCnt.1.9f:
>
> $ ls /var/lib/BackupPC/pc/*/*/refCnt/poolCnt
>
> $ ls: cannot access '/var/lib/BackupPC/pc/*/*/refCnt/poolCnt': No such file
> or directory
> ...

Even if there were such files, your ls command wouldn't list them
because it looks for files named 'poolCnt' not 'poolCnt*'. :(

I recommend getting the hang of the 'find' utility, but be warned that
in my experience it's an acquired taste.  You could always pipe the
output of ls -lR into a pager (or grep) but the right utility will be
more efficient.

> I also can?t see what caused this to happen in the first place. ...
> Can anyone give any insight in to what is going on here ...

Did a machine experience a power failure, crash, or something like
that?  Perhaps someone restarted BackupPC at an inconvenient time?
Did you experiment with compression?  Manually fiddle with the pool?

> ... how I might be able to find out which file is missing?

BackupPC_poolCntPrint may give some useful information, see e.g.

https://sourceforge.net/p/backuppc/mailman/message/36379446/

The syntax isn't always obvious so here's an example using BackupPC_ls
much as Craig suggests in his post linked above, to list md5sums on a
box backed up here:

backuppc@server:$ /usr/local/BackupPC/bin/BackupPC_ls -R -h host44 -n 2072 -s Config / | head
/:
-rw-------       0/0          0 2013-06-27 13:09:27  /.pwd.lock (d41d8cd98f00b204e9800998ecf8427e)
drwxr-xr-x       0/0          0 2021-09-03 14:52:44  /ConsoleKit/
-rw-r--r--       0/0       1467 2019-01-18 15:26:02  /GeoIP.conf (fe5e3fdcd6ad284bb57e63fe41f4e553)
drwxr-xr-x       0/0          0 2015-05-03 19:19:43  /ImageMagick/
drwxr-xr-x       0/0          0 2024-03-20 07:44:44  /ImageMagick-6/
-rw-r--r--       0/0       4954 2021-01-25 18:10:07  /Muttrc (6793ae4ed35d7fa37a57e2b1fd1507e8)
[very big snip]

You'd grep your output for a06340ddb577fd02a04bda2998a8d246.

There might be more scripts kicking around on the mailing list to do
things like this, I can't recall, but sometimes random scripts need
bringing up to date.  So unless you like puzzles I'd recommend using
the tools provided before going off the reservation.

-- 

73,
Ged.

[BackupPC-users] Missing pool files

[Jamie B. <ja...@ib...>]

Hello



Out of the blue on the 24th September, my log showed:



2025-09-24 01:00:20  admin1 : BackupPC_refCountUpdate: missing pool file
817a2e4df8f74f2a73ba4ea8ef2a0241 count 7

2025-09-24 01:00:43  admin : BackupPC_refCountUpdate: missing pool file
0ea033c2c09692ae539cac4a106803e9 count 18

2025-09-24 01:00:45  admin1 : BackupPC_refCountUpdate: missing pool file
915ee38d27ca01472a979a084031689e count 19

2025-09-24 01:00:47  admin1 : BackupPC_refCountUpdate: missing pool file
9393a0f2fac7834aa850f572135694c9 count 18

2025-09-24 01:00:52  admin1 : BackupPC_refCountUpdate: missing pool file
97e40662de06f70ace0e9497d5ebfc7f count 2

2025-09-24 01:00:56  admin : BackupPC_refCountUpdate: missing pool file
1aa4e46a56b5bae7ac6cab0f47662b3d count 17

2025-09-24 01:01:03  admin1 : BackupPC_refCountUpdate: missing pool file
a06340ddb577fd02a04bda2998a8d246 count 1

2025-09-24 01:01:10  admin1 : BackupPC_refCountUpdate: missing pool file
a6e7085e4fbe48bf605c226cba3a8a0b count 19

2025-09-24 01:01:32  admin1 : BackupPC_refCountUpdate: missing pool file
b95dcca650ac8918e8ca3c126698ae58 count 2

2025-09-24 01:01:33  admin : BackupPC_refCountUpdate: missing pool file
3bff93ec664e6bfa38febd3aca108159 count 20

2025-09-24 01:01:51  admin : BackupPC_refCountUpdate: missing pool file
49f985a7ed2889c7b9352627cba69b4e count 19

2025-09-24 01:01:54  admin : BackupPC_refCountUpdate: missing pool file
4bd09fabd304d2a20ed52f9609f5a397 count 55

2025-09-24 01:01:54  admin1 : BackupPC_refCountUpdate: missing pool file
cdfaf4d9e80d464f4b9098d9ebaa5a24 count 17

2025-09-24 01:02:02  admin : BackupPC_refCountUpdate: missing pool file
53c090a28927806d81ca8e63d3bf4a7a count 19

2025-09-24 01:02:21  admin : BackupPC_refCountUpdate: missing pool file
64377d23b54c896bdee4b83df9006ac4 count 17

2025-09-24 01:02:25  admin : BackupPC_refCountUpdate: missing pool file
6919c47f84b7d5a40c453435076d11a9 count 2

2025-09-24 01:02:30  admin1 : BackupPC_refCountUpdate: missing pool file
ee88a6fce8449682d6b4465f89b447ec count 19

2025-09-24 01:03:26  admin1 : BackupPC_refCountUpdate total errors: 9



Logs prior to this show no missing pool files and logs after this show a
decreasing number of missing files. Only one has remained:



2025-10-07 01:00:48  admin1 : BackupPC_refCountUpdate: missing pool file
a06340ddb577fd02a04bda2998a8d246 count 1



Curious which backup this pertains to, I found and run the script mentioned
here (https://sourceforge.net/p/backuppc/mailman/message/58712645/) but the
output only shows:



Search for missing pool files started. Logs will be written to
/home/backuppc/find_missing_pool_files_20251007-09:10.log

a0 - 1 = 9f

/var/lib/BackupPC/pc/*/*/refCnt/poolCnt.1.9f

a0 - 1 = 9f

/var/lib/BackupPC/pc/*/*/refCnt/poolCnt.1.9f



No log file is created and I cannot find a file named poolCnt.1.9f:



$ ls /var/lib/BackupPC/pc/*/*/refCnt/poolCnt

$ ls: cannot access '/var/lib/BackupPC/pc/*/*/refCnt/poolCnt': No such file
or directory



I also can’t see what caused this to happen in the first place. The logs
don’t show any other errors.



Can anyone give any insight in to what is going on here and how I might be
able to find out which file is missing?



Kind regards,

*Jamie*
ib3 Limited

------------------------------


[image: logo]


*Jamie Burchell*
Senior Web Developer


01732 449974
ja...@ib...


*ib3 Limited*
2 Lyons Wharf, Lyons Crescent,
Tonbridge, Kent TN9 1EX

Main 01732 449970
www.ib3.uk

<https://www.facebook.com/ib3co/>
<https://www.linkedin.com/company/263186/>   <http://twitter.com/ib3uk>
------------------------------


This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of ib3 Limited. If you are not the intended recipient of
this email, you must neither take any action based upon its contents, nor
copy or show it to anyone. Please contact the sender if you believe you
have received this email in error.


*ib3 Limited is a company registered in England. Registered number:
3734612.Registered office: 2 Lyons Wharf, Lyons Crescent, Tonbridge, Kent
TN9 1EX, United Kingdom.*

Re: [BackupPC-users] Backing up Windows shares.

[Falko T. <ne...@tr...>]

G.W. Haywood schrieb am 28.09.25 um 13:49:
> Comments welcome.  I can't test on anything more recent than a Win7 VM
> which I have kicking around from the days - sorry, years - when I
> supported Windows installations, thankfully now long past.

That's what I did more than five years ago, too - but we didn't use 
samba client for Windows backups with BackupPC, but this:

https://www.michaelstowe.com/backuppc/

and I remember it working very well. Especially that it did
use shadow copies. I think we used Win10 these days.

But later we switched to Urbackup client.

If you don't have recent Windows versions at hand ...
Usually it is possible to get some Windows developer edition VMs.
Ah, we did use this: https://github.com/xdissent/ievms/

Maybe some newer fixes are in the forks or pull requests like:
https://github.com/xdissent/ievms/pull/343

Just my 2¢
Falko

Re: [BackupPC-users] Backing up Windows shares.

[Michael S. <mic...@me...>]

On 2025-09-28 04:49, G.W. Haywood wrote:
> Hi there,
> 
> For many years BackupPC has offered the option of using using Samba's
> smbclient to back up Windows SMB shares.
> 
> Unfortunately it seems that in this use case there are multiple issues
> with smbclient, see for example from January 2019:
> 
> https://github.com/backuppc/backuppc/issues/252
> 
> Windows junctions can make it impossible to exclude things you want to
> exclude; one particular problem with smbclient is that excludes don't
> seem to work properly, and haven't done for some time.
> 
> The 'regex' option for smbclient's tarmode is (a) broken since about
> Samba 4.13, five years or more ago, and (b) deprecated in any case by
> the Samba team since Samba 4.2 was released in March 2015 - so I don't
> know if the option is ever likely to be fixed; one day it might simply
> disappear.  Although Samba is very actively developed, a regression
> reported on the Samba bugzilla in November 2022:
> 
> https://bugzilla.samba.org/show_bug.cgi?id=15229
> 
> has generated no response.  That may be because the 'r' option is, as
> noted, now deprecated by Samba.  A 'wontfix' would have helped, but I
> suspect that the lack of response from BackupPC developers to queries
> in the past by Samba team members might figure to some extent in the
> equation.  For the avoidance of doubt, I'm now the BackupPC maintainer
> and I want somehow to get this fixed for Windows users.
> 
> So I got to wondering if anyone here is using CIFS to mount Windows 10
> or Windows 11 SMB shares for BackupPC to read?  I'm sure it's possible
> (apparently a lot of people using older NAS devices do it) but I don't
> personally know anyone who's doing it.  It needs some administration
> to get it working because it's not enabled by default in later Windows
> versions, largely I think because of the potential security issues.
> 
> It seems to me that even though like anything else CIFS has security
> issues, CIFS access could be restricted to the BackupPC server which
> should eliminate the vast majority.  Using a remote mount would allow
> BackupPC to use tar instead of smbclient, thus eliminating the problem
> with excludes and potentially a bunch of other issues using smbclient.
> 
> Yet another alternative might be to use a different SMB client, but I
> know nothing about such clients except that at least one exists.  This
> would probably require quite a lot of development in BackupPC which to
> me makes it an unlikely candidate at least in the near term (i.e. the
> next few years) unless someone here wants to step up and do the work.
> 
> Comments welcome.  I can't test on anything more recent than a Win7 VM
> which I have kicking around from the days - sorry, years - when I
> supported Windows installations, thankfully now long past.

It's not just Windows junctions, but Windows file semantics that make 
smbclient (and CIFS) a second choice.  There are at least a couple 
rsync/Windows solutions out there, both of which use shadow volumes to 
align with Windows' own backup paradigms.  They also have the advantage 
of not requiring Windows shares at all, which (as you've noted) have 
fallen out of fashion.

There have been a handful of people who reported mounting Windows shares 
locally to back those up via BackupPC, but I recall the experience being 
slow and brittle and I'd be a little surprised if anybody continued with 
it.

[BackupPC-users] Backing up Windows shares.

[G.W. H. <ba...@ju...>]

Hi there,

For many years BackupPC has offered the option of using using Samba's
smbclient to back up Windows SMB shares.

Unfortunately it seems that in this use case there are multiple issues
with smbclient, see for example from January 2019:

https://github.com/backuppc/backuppc/issues/252

Windows junctions can make it impossible to exclude things you want to
exclude; one particular problem with smbclient is that excludes don't
seem to work properly, and haven't done for some time.

The 'regex' option for smbclient's tarmode is (a) broken since about
Samba 4.13, five years or more ago, and (b) deprecated in any case by
the Samba team since Samba 4.2 was released in March 2015 - so I don't
know if the option is ever likely to be fixed; one day it might simply
disappear.  Although Samba is very actively developed, a regression
reported on the Samba bugzilla in November 2022:

https://bugzilla.samba.org/show_bug.cgi?id=15229

has generated no response.  That may be because the 'r' option is, as
noted, now deprecated by Samba.  A 'wontfix' would have helped, but I
suspect that the lack of response from BackupPC developers to queries
in the past by Samba team members might figure to some extent in the
equation.  For the avoidance of doubt, I'm now the BackupPC maintainer
and I want somehow to get this fixed for Windows users.

So I got to wondering if anyone here is using CIFS to mount Windows 10
or Windows 11 SMB shares for BackupPC to read?  I'm sure it's possible
(apparently a lot of people using older NAS devices do it) but I don't
personally know anyone who's doing it.  It needs some administration
to get it working because it's not enabled by default in later Windows
versions, largely I think because of the potential security issues.

It seems to me that even though like anything else CIFS has security
issues, CIFS access could be restricted to the BackupPC server which
should eliminate the vast majority.  Using a remote mount would allow
BackupPC to use tar instead of smbclient, thus eliminating the problem
with excludes and potentially a bunch of other issues using smbclient.

Yet another alternative might be to use a different SMB client, but I
know nothing about such clients except that at least one exists.  This
would probably require quite a lot of development in BackupPC which to
me makes it an unlikely candidate at least in the near term (i.e. the
next few years) unless someone here wants to step up and do the work.

Comments welcome.  I can't test on anything more recent than a Win7 VM
which I have kicking around from the days - sorry, years - when I
supported Windows installations, thankfully now long past.

-- 

73,
Ged.

Re: [BackupPC-users] Why is my v3 pool growing?

[Matthew P. <ma...@co...>]

On Wed, Sep 17, 2025 at 10:36 AM G.W. Haywood <ba...@ju...>
wrote:

> Firstly, I'd be inclined to check the numbers in some way other than
> by looking at the graph. :)  It wouldn't be the first time that some
> graph or other didn't perfectly reflect the underlying data.  It may
> be helpful to know exactly how much data we're talking about in the V3
> pool (as the graphs aren't terribly precise) and also how many files.
> It would be good to eliminate somehow any possibility that BackupPC
> isn't the culprit, since BackupPC's graph just shows you approximately
> how much data is in the pool directories, it doesn't actually know how
> it got there.
>

I can't find clear documentation about how to identify v3 pool files.  It
seems to be sort of implied in the docs that they are the single-character
directories in cpool?   I'm guessing based on the fact that the section on
v4 attrib files only mentions two-character directories at the top of the
tree.
If that's the case then it's about 15% of the total (1.2G out of 7.7G).


> Secondly, presumably you have
> $Conf{PoolV3Enabled} = 1;
>

I do.

In that case when it's doing a backup, BackupPC will of course use the
> V3 pool and matching data found there.  Like you, I wouldn't expect
> *new* V3 pool/cpool data to be written but I haven't done any tests at
> all to see what might happen under oddball circumstances[*].  I see
> setting $Conf{PoolV3Enabled} = 1; in the nature of stop-gap measure,
> and I recommend setting it to 0 ASAP.  Of course that supposes that
> all your V3 backups are expired or considered to be of no more use.
>

So I looked... the paths under `cpool/?` (single-character pool
directories)  with the most recent mod-times are:
1) directories (e.g. `cpool/f/6/0`)
2) most recently modified 105 days ago

That is definitely within a few days of when I ran BackupPC_migrateV3toV4,
if not that very day.  So I think that confirms I'm not getting new v3
backups.

Which really just leaves some kind of anomaly or variation in the data
collection for the RRD or data extraction from the RRD.   Again, I'm not
really concerned about this, just curious what's up.  If I find myself with
all kinds of free time (HA!) maybe I'll dig into the data tables just to
make sure which side of that line the issue is on.

Thanks for the assist!
  Matt


>

Re: [BackupPC-users] Why is my v3 pool growing?

[G.W. H. <ba...@ju...>]

Hi there,

On Wed, 17 Sep 2025, Matthew Pounsett wrote:

Re: Why is my v3 pool growing?

> ... the reported size of the v3 pool seems to rise and fall.  Since
> I don't think new v3 backups should be getting created?that would be
> astonishing?I'm at a loss to explain it.  And I'm really, really
> curious what's up.  I expect it to shrink over time as very-old
> backups expire, and eventually just reach zero.  I did not expect to
> see it ever grow.
>
> Does anyone have an idea of what's happening here?

Firstly, I'd be inclined to check the numbers in some way other than
by looking at the graph. :)  It wouldn't be the first time that some
graph or other didn't perfectly reflect the underlying data.  It may
be helpful to know exactly how much data we're talking about in the V3
pool (as the graphs aren't terribly precise) and also how many files.
It would be good to eliminate somehow any possibility that BackupPC
isn't the culprit, since BackupPC's graph just shows you approximately
how much data is in the pool directories, it doesn't actually know how
it got there.

Secondly, presumably you have

$Conf{PoolV3Enabled} = 1;

in your configuration file.

In that case when it's doing a backup, BackupPC will of course use the
V3 pool and matching data found there.  Like you, I wouldn't expect
*new* V3 pool/cpool data to be written but I haven't done any tests at
all to see what might happen under oddball circumstances[*].  I see
setting $Conf{PoolV3Enabled} = 1; in the nature of stop-gap measure,
and I recommend setting it to 0 ASAP.  Of course that supposes that
all your V3 backups are expired or considered to be of no more use.
If V3 backups still exist and you set $Conf{PoolV3Enabled} = 0; then I
guess that the worst that will happen is that files get copied to the
V4 pool instead of the existing V3 copies being used.  That will take
some extra time and probably storage space, so we're back to asking
how much data (and perhaps how much free storage?), but that will only
happen once.  Any remaining V3 backups would *not* then be cleaned up
by BackupPC itself, and would need to be deleted manually if required.

It wouldn't be a huge stretch of the imagination to think that the V3
pool might sometimes be used when it's not expected.  I'd want to know
under what circumstances.  To find out if this happens and if so when,
if it were my system I'd probably for example run cron jobs to dump
directory listings between backup runs, to view at my (ho-ho) leisure.
Maybe I'd also add some extra logging statements to the BackupPC code,
there are already a few which might give you some ideas, for example
in sub ScanAndCleanV3Pool() in BackupPC_nightly.

[*] For example hash collisions, system restarts, old tmpfiles?

-- 

73,
Ged.

[BackupPC-users] Why is my v3 pool growing?

[Matthew P. <ma...@co...>]

This is not a Problem...I'm just trying to understand some behaviour.

A few months ago I ran the V3->V4 conversion because it appeared like that
hadn't been done when our v4 upgrade was done.  That got me brand new data
in my graphs tracking the remaining v3 CPool.  Yay! [big-thumbs-up]

However, I've noticed that the reported size of the v3 pool seems to rise
and fall.  Since I don't think new v3 backups should be getting
created—that would be astonishing—I'm at a loss to explain it.  And I'm
really, really curious what's up.   I expect it to shrink over time as
very-old backups expire, and eventually just reach zero.  I did not expect
to see it ever grow.

Does anyone have an idea of what's happening here?

I don't see any attachments in my email history for the list, so I'm
assuming an image attachment would be stripped.  Here's a URL to a
screenshot to see what I mean:
<
https://www.dropbox.com/scl/fi/t4woykfalnr0h9yx5hwth/Screenshot-2025-09-16-at-12.01.18.png?rlkey=h4qfrfjahycyw7vacvgq6e5du&dl=0
>

Re: [BackupPC-users] Delete files from backups/pool?

[G.W. H. <ba...@ju...>]

Hi there,

On Tue, 2 Sep 2025, Steve Richards wrote:

> > Not really recommended but if you just deleted the huge files from the
> > pool then you'd get some error messages when e.g. nightly checks were
> > run, but I think that should be about the extent of the inconvenience.
> 
> Not sure I want to try that if it's not recommended but, if I did, how 
> would I go about finding and deleting the files given that the historic 
> guidance is now obsolete?

You can use the script BackupPC_ls to find the files in the pool.  The
script is in BackupPC's /bin/ directory.  Below is an example, which I
ran on our pool on backup server 'piplus' to list directory 'cups' in
share 'Config' on our host 'alpha'.  Years ago I configured the share
'Config' to point to the '/etc/' directory on this host - every host I
back up has a BackupPC 'Config' share.

8<----------------------------------------------------------------------
piplus:# >>> su - backuppc
backuppc@piplus: $ /usr/local/BackupPC/bin/BackupPC_ls -h alpha -n 2004 -s Config cups
cups:
-rw-r--r--       0/0      27408 2024-09-27 13:34:52  cups/cups-browsed.conf (0b812d29a7f82f521a3c384ba53f831c)
-rw-r--r--       0/0      27303 2019-04-17 10:04:46  cups/cups-browsed.conf~ (b2cbc92bec253cfd9c7fadb2c018cd66)
-rw-r--r--       0/0       2923 2019-08-21 08:43:13  cups/cups-files.conf (39776f9574026852ef283113ac2485d8)
-rw-r-----       0/7       4669 2022-12-02 15:16:05  cups/cupsd.conf (6dfdf833d9ee3da91ecc0c46c415cf50)
-rw-r--r--       0/0       6402 2020-01-13 17:37:56  cups/cupsd.conf.O (1faa528b0f83af98e55510e5945e0298)
drwxr-xr-x       0/0          0 2019-08-21 08:43:13  cups/interfaces/
drwxr-xr-x       0/7          0 2020-12-18 15:33:01  cups/ppd/
-rw-------       0/7       2214 2025-08-31 00:00:42  cups/printers.conf (316f85ea58575570bdda0773e81d811c)
-rw-------       0/7       2214 2025-08-30 00:00:36  cups/printers.conf.O (316f85ea58575570bdda0773e81d811c)
-rw-r--r--       0/0        240 2020-01-13 17:38:05  cups/raw.convs (cb64c71ae2fd35a2ff07e54a6098eecf)
-rw-r--r--       0/0        211 2020-01-13 17:38:05  cups/raw.types (b507ca634c5be3c42db5700ec745ac0d)
-rw-r--r--       0/0        142 2022-12-02 15:17:37  cups/snmp.conf (47b8f1c3fecdc44e3d1fdee4b9eeb3f5)
drwx------       0/7          0 2023-05-03 10:36:28  cups/ssl/
-rw-r-----       0/7        388 2025-08-31 00:00:43  cups/subscriptions.conf (0692d940f7ea1701919938333a8a0ab8)
-rw-r-----       0/7         94 2025-08-31 00:00:09  cups/subscriptions.conf.O (b0b6799bfe8fea62737eb7deb2037bfa)
backuppc@piplus: $ ls -l /var/lib/BackupPC/cpool/30/6e/316f85ea58575570bdda0773e81d811c
-r--r--r-- 1 backuppc backuppc 880 Jul  9 01:32 /var/lib/BackupPC/cpool/30/6e/316f85ea58575570bdda0773e81d811c
backuppc@piplus: $ /usr/local/BackupPC/bin/BackupPC_zcat /var/lib/BackupPC/cpool/30/6e/316f85ea58575570bdda0773e81d811c
# Printer configuration file for CUPS ...
...
...
backuppc@piplus: $
logout
8<----------------------------------------------------------------------

As you can see BackupPC_ls gives the names of the files in the pool.
The file names are hashes of the file content - hexadecimal numbers.
For better efficiency of directory searches the files are stored in
two levels of directories.  Each level has 128 directories, named as
00, 02, 04, ... fa, fc, fe.  As there are 128 (not 256) directories,
the first four characters of each filename are converted to 'even'
numbers and these are used as the path to the directory where the file
will be stored.  In my example I changed '31' to '30' and '6f' to '6e'
and gave that path to BackupPC_zcat to uncompress the file to stdout.
It's all very straightforward when you get the hang of it, but do be
careful please if you do this with non-text files, as sending binary
data to a terminal can leave it in a weird state.  I find that typing
the 'reset' command will usually fix it. :)

> ... Would deleting all backups that included/referenced the files
> cause them to be cleaned out of the pool (and the space recovered)
> automatically ...

Yes, the nightly routines will do that.  As they are time-consuming,
BackupPC can be configured to check just a part of the pool each night
so depending on your configuration options it may take a few days to
recover all the space.  See config.pl for details.

-- 

73,
Ged.

Re: [BackupPC-users] Delete files from backups/pool?

[<Mat...@gm...>]

Hello Steve,

There are some differences in storage layout between BackupPC V3 and V4. Because this you can't
apply a 20 year old documentation 😁️.

In BackupPC V4 all files are stored only once in ../pool/??/??/ or in ../cpool/??/??/.
You can delete this file in [c]pool but you will harvest on "admin : BackupPC_refCountUpdate:
missing pool file <hex-number>" in your log file.

What you have to do to remove the file (e.g. for bash):
   1. find the name of the poolfile
       - hex=$(sudo -u backuppc $bpcBin/BackupPC_attribPrint
         /var/lib/backuppc/pc/<yourHost>/<bckNr>/f<share>/<path>/attrib_* | grep -A2 <yourFileName> | grep
         -A2 "{" | grep digest | awk '{print $3}')
       - hex=${hex:1:${#hex}-3}
       - hex1=$(printf "%02x" $(( 0x${hex:0:2}/2*2)))
       - hex2=$(printf "%02x" $(( 0x${hex:2:2}/2*2)))
   2. remove the pool-file
         1. rm -f ../[c]pool/$hex1/$hex2/$hex

I've never done it - so be carefully and test it :)
What I'd propose - and I'll do this soon - would be to find a small pool-file and create a hardlink.
So your 8GB file is linked to this small pool-file and is using no additional size.

Br
Matthias

Am Montag, dem 01.09.2025 um 11:37 +0100 schrieb Steve Richards:
> I have accidentally allowed BackupPC to backup some very large files (3 x ~8GB) from one of my
> hosts. I don't need them backed up and would like to get rid of them from the backup storage. If I
> understand correctly that would mean removing stuff from BackupPC's pool system. Can I do that
> safely? 
>  
>  I found a very old post (20+ years ago) about deleting from the pool but I know BackupPC has had
> at least one major revision in that time and I don't know whether the advice back then is still
> appropriate with current versions. The post says:
>  
>  # cd __TOPDIR__/pc/
>  # find . -name *.mp3
>  <you should see a list of your .mp3 files here. if it looks like a good list
>  (i.e. you didn't typo something that will be painful to fix), go to the next
>  step.>
>  # find . -name *.mp3 -exec rm -f {} \;
>  
>  On my installation (v4.4.0) that doesn't seem to find any files matching the search pattern
> (*.raw in my case), or indeed any identifiable files, which makes me wonder if the pooling
> mechanism and structure might have changed. In any case I don't want to mess with anything low-
> level without up to date advice. The "rogue" files were first backed-up only a couple of days ago,
> so I would be happy to delete all backups from that date if that would help.
>  
>  Thanks.
> _______________________________________________
> BackupPC-users mailing list
> Bac...@li...
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Delete files from backups/pool?

[Steve R. <bp...@bo...>]

Thanks for the response.

> Not really recommended but if you just deleted the huge files from the
> pool then you'd get some error messages when e.g. nightly checks were
> run, but I think that should be about the extent of the inconvenience.

Not sure I want to try that if it's not recommended but, if I did, how 
would I go about finding and deleting the files given that the historic 
guidance is now obsolete?

> You can safely delete entire backups using the Web interface.  You can
> safely use the script BackupPC_backupDelete for more control to delete
> shares or directories, but not to the level of individual files. The
> script is in BackupPC's /bin/ directory.  Like most BackupPC scripts,
> it should be run by user backuppc.  If you run it with no arguments it
> will give help on usage.  To get the hang of using it, maybe try it on
> a backup that you aren't especially fond of.

OK, thanks. Would deleting all backups that included/referenced the 
files cause them to be cleaned out of the pool (and the space recovered) 
automatically, or is there a further step?