dclib-devel — A list for developers to discuss issues

[Dclib-devel] [dclib-devel] loss_multiclass_log_per_pixel_weighted and 2d binary classification

[Eloi Du B. <elo...@gm...>]

Hi,

I would like to make a classifier over a 2D matrix with 2 classes defective
/ non defective.

Do you recommend using 1 class (0) + not_a_label or 2 classes in {0, 1} ?

Thanks,

Re: [Dclib-devel] [dclib-devel] DLib dnn precision

[Davis K. <dav...@gm...>]

Everything is in float.  Double precision is much slower on GPUs and is not
needed.

[Dclib-devel] [dclib-devel] DLib dnn precision

[Eloi Du B. <elo...@gm...>]

Hi,

I just wanted to know what precision you use for neural nets tensors.
Is there an interest in using doubles instead of floats?

Thanks,

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Re: [Dclib-devel] [dclib-devel] add_skip_layer to add to layers

[Davis K. <dav...@gm...>]

Yes, basically.  But you don't need to use add_skip_layer, just use skip3,
it's way more concise.  There is an example in the second dnn introduction
example program.

Re: [Dclib-devel] [dclib-devel] add_skip_layer to add to layers

[Eloi Du B. <elo...@gm...>]

This seems to be working:
    template <template <typename> class Block1, template <typename> class
Block2, typename SUBNET>
    using merge_add =
dlib::add_prev2<Block1<dlib::add_skip_layer<dlib::tag3,
dlib::tag2<Block2<dlib::tag3<SUBNET>>>>>>;


2018-02-10 18:59 GMT-06:00 Eloi Du Bois <elo...@gm...>:

> Sorry, this should actually make it, what do you think?:
>     template <template <typename> class Block, typename SUBNET>
>     using merge_add = dlib::add_prev2<Block<dlib::add_skip_layer<dlib::tag1,
> dlib::tag2<dlib::tag1<SUBNET>>>>>;
>
> It could be nice to add it into dlib.
>
> 2018-02-10 18:48 GMT-06:00 Eloi Du Bois <elo...@gm...>:
>
>> Could this be it?
>>     template <template <typename> class Block, typename SUBNET>
>>     using merge = dlib::add_prev2<dlib::add_skip_layer<dlib::tag2,
>> Block<dlib::htan<dlib::tag2<SUBNET>>>>>;
>>
>> Thanks,
>>
>> 2018-02-10 18:44 GMT-06:00 Eloi Du Bois <elo...@gm...>:
>>
>>> Hi,
>>>
>>> I'd like to add two layers, both using the same input.
>>> This is different than using add_prev as when using add_prev, one of the
>>> path is a chain.
>>>
>>> How could I do this?
>>> I suspect that add_skip_layer would solve this, but I'm not sure how.
>>>
>>> Many thanks for any answer.
>>>
>>>
>>
>

Re: [Dclib-devel] [dclib-devel] add_skip_layer to add to layers

[Eloi Du B. <elo...@gm...>]

Sorry, this should actually make it, what do you think?:
    template <template <typename> class Block, typename SUBNET>
    using merge_add =
dlib::add_prev2<Block<dlib::add_skip_layer<dlib::tag1,
dlib::tag2<dlib::tag1<SUBNET>>>>>;

It could be nice to add it into dlib.

2018-02-10 18:48 GMT-06:00 Eloi Du Bois <elo...@gm...>:

> Could this be it?
>     template <template <typename> class Block, typename SUBNET>
>     using merge = dlib::add_prev2<dlib::add_skip_layer<dlib::tag2,
> Block<dlib::htan<dlib::tag2<SUBNET>>>>>;
>
> Thanks,
>
> 2018-02-10 18:44 GMT-06:00 Eloi Du Bois <elo...@gm...>:
>
>> Hi,
>>
>> I'd like to add two layers, both using the same input.
>> This is different than using add_prev as when using add_prev, one of the
>> path is a chain.
>>
>> How could I do this?
>> I suspect that add_skip_layer would solve this, but I'm not sure how.
>>
>> Many thanks for any answer.
>>
>>
>

Re: [Dclib-devel] [dclib-devel] add_skip_layer to add to layers

[Eloi Du B. <elo...@gm...>]

Could this be it?
    template <template <typename> class Block, typename SUBNET>
    using merge = dlib::add_prev2<dlib::add_skip_layer<dlib::tag2,
Block<dlib::htan<dlib::tag2<SUBNET>>>>>;

Thanks,

2018-02-10 18:44 GMT-06:00 Eloi Du Bois <elo...@gm...>:

> Hi,
>
> I'd like to add two layers, both using the same input.
> This is different than using add_prev as when using add_prev, one of the
> path is a chain.
>
> How could I do this?
> I suspect that add_skip_layer would solve this, but I'm not sure how.
>
> Many thanks for any answer.
>
>

[Dclib-devel] [dclib-devel] add_skip_layer to add to layers

[Eloi Du B. <elo...@gm...>]

Hi,

I'd like to add two layers, both using the same input.
This is different than using add_prev as when using add_prev, one of the
path is a chain.

How could I do this?
I suspect that add_skip_layer would solve this, but I'm not sure how.

Many thanks for any answer.

Re: [Dclib-devel] [dclib-devel] DNN: Add previous layer

[Eloi Du B. <elo...@gm...>]

Ok, that answer my question, great, thanks.

2018-02-10 15:10 GMT-06:00 Davis King <dav...@gm...>:

> add_prev1 walks the network backward until it finds a tag1 layer.  So that
> means that as long as Block doesn't have any tag1 layers in it then it's
> fine.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] [dclib-devel] DNN: Add previous layer

[Davis K. <dav...@gm...>]

add_prev1 walks the network backward until it finds a tag1 layer.  So that
means that as long as Block doesn't have any tag1 layers in it then it's
fine.

[Dclib-devel] [dclib-devel] DNN: Add previous layer

[Eloi Du B. <elo...@gm...>]

Hi,

Let's say I define:
    template <template <typename> class Block, typename SUBNET>
    using merge = dlib::add_prev1<Block<dlib::tag1<SUBNET>>>;

Is it ok to use this recursively, like:
using Net = dlib::relu<merge<block1, merge<block2, InputMat>>>;

I mean, is it going to be a problem because I use several times tag1?

If this is not correct how can I do? I just want to add 3 layers together.

Thanks,

Re: [Dclib-devel] Distributed learning

[Davis K. <dav...@gm...>]

The structural svm training tools have such a mode.  And the dnn_trainer
can use multiple GPUs at a time. Other than that no, I'm not going to do
this.

[Dclib-devel] Distributed learning

[Eloi Du B. <elo...@gm...>]

Hi,

Just want to know, any future plan on implementing a tool to do some
distributed learning?
Do you have advises on how to do this with dlib?

Thanks,

Re: [Dclib-devel] Drawing dlib's SVM hyperplane

[Eloi Du B. <elo...@gm...>]

Ok, thanks, that at least gives me where to look at.

2018-01-29 13:55 GMT-06:00 Davis King <dav...@gm...>:

> Get the decision_function.  It's all in there.  It's also discussed in the
> documentation for each trainer.
>
> On Mon, Jan 29, 2018 at 2:48 PM, Eloi Du Bois <elo...@gm...>
> wrote:
>
>> And when using a linear svm c trainer, where is stored the equation
>> parameter?
>> I'm expecting that there is a a 'a' and 'b' in the form y=ax+b
>>
>>
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon> Virus-free.
>> www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
>> <#m_-2252614085694331993_m_-115297011536388346_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>
>> 2018-01-29 13:39 GMT-06:00 Davis King <dav...@gm...>:
>>
>>> I would use matplotlib.  I don't have any code for it though.
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Dclib-devel mailing list
>>> Dcl...@li...
>>> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>>>
>>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Dclib-devel mailing list
>> Dcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>>
>>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] Drawing dlib's SVM hyperplane

[Davis K. <dav...@gm...>]

Get the decision_function.  It's all in there.  It's also discussed in the
documentation for each trainer.

On Mon, Jan 29, 2018 at 2:48 PM, Eloi Du Bois <elo...@gm...>
wrote:

> And when using a linear svm c trainer, where is stored the equation
> parameter?
> I'm expecting that there is a a 'a' and 'b' in the form y=ax+b
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
> <#m_-115297011536388346_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> 2018-01-29 13:39 GMT-06:00 Davis King <dav...@gm...>:
>
>> I would use matplotlib.  I don't have any code for it though.
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Dclib-devel mailing list
>> Dcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>>
>>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] Drawing dlib's SVM hyperplane

[Eloi Du B. <elo...@gm...>]

And when using a linear svm c trainer, where is stored the equation
parameter?
I'm expecting that there is a a 'a' and 'b' in the form y=ax+b

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2018-01-29 13:39 GMT-06:00 Davis King <dav...@gm...>:

> I would use matplotlib.  I don't have any code for it though.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] Drawing dlib's SVM hyperplane

[Davis K. <dav...@gm...>]

I would use matplotlib.  I don't have any code for it though.

[Dclib-devel] Drawing dlib's SVM hyperplane

[Eloi Du B. <elo...@gm...>]

Hi,

Do you have any script / code / advise on how to make a 3d plot of the
hyperplane of an trained SVM using Dlib?

Thanks,


<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Re: [Dclib-devel] Clean on model during training

[Davis K. <dav...@gm...>]

Yes

Re: [Dclib-devel] Clean on model during training

[Eloi Du B. <elo...@gm...>]

Ok, is the clean method reducing the size of the saved NN?

Thanks,

2018-01-07 16:23 GMT-06:00 Davis King <dav...@gm...>:

> Yes, that’s fine.
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>

Re: [Dclib-devel] Clean on model during training

[Davis K. <dav...@gm...>]

Yes, that’s fine. 

[Dclib-devel] Clean on model during training

[Eloi Du B. <elo...@gm...>]

Hi,

When training a NN, I am using several passes of training, and in between,
I would like to save the model. I see that it's advised to use the clean
method, but is it safe as I'm going to continue the training in another
pass?

Here is a pseudo code:

while(k < 100)
{
  train(model, k);
  model.clean();
  serialize(model) >> "file.dat";
}

Thanks,

Re: [Dclib-devel] [dclib-devel] loss_mean_squared_per_pixel_ and gradient

[Davis K. <dav...@gm...>]

Every function and class in dlib is documented.  When you have questions
about what something does you should refer to the documentation.  It's all
explained there.

On Thu, Dec 14, 2017 at 7:09 PM, Eloi Du Bois <elo...@gm...>
wrote:

> I'm trying to make an 2d auto-encoder using loss_mean_squared_per_pixel.
> It is really strange, I am getting a loss of 10e-5 on a training set, but
> then, when I apply a forward pass on my validation set, which contains
> something very very similar to the training set, I get an error of 0.1. I
> think the problem is on my side, but I see something that I would like to
> understand in this loss function:
> In file loss.h +2421
> It is written:
> float* const g = grad.host();
>
> On other functions I see:
> float* const g = grad.host_write_only();
>
> Could this be a problem? What is the difference?
>
> Just want to make sure the GPU gradient is well updated.
>
> Thanks,
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
> <#m_-8943620989895474723_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

[Dclib-devel] [dclib-devel] loss_mean_squared_per_pixel_ and gradient

[Eloi Du B. <elo...@gm...>]

I'm trying to make an 2d auto-encoder using loss_mean_squared_per_pixel.
It is really strange, I am getting a loss of 10e-5 on a training set, but
then, when I apply a forward pass on my validation set, which contains
something very very similar to the training set, I get an error of 0.1. I
think the problem is on my side, but I see something that I would like to
understand in this loss function:
In file loss.h +2421
It is written:
float* const g = grad.host();

On other functions I see:
float* const g = grad.host_write_only();

Could this be a problem? What is the difference?

Just want to make sure the GPU gradient is well updated.

Thanks,

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Re: [Dclib-devel] [dclib-devel] input_rgb_image and division by 256

[Eloi Du B. <elo...@gm...>]

Ah no sorry, avg_* are float values, so it doesn't matter.

2017-12-08 16:06 GMT-06:00 Eloi Du Bois <elo...@gm...>:

> It can have a very low influence when making an auto encoder, as you would
> have a 7 bits input instead of 8 if I'm right. It's minor I agree.
>
> 2017-12-08 16:03 GMT-06:00 Davis King <dav...@gm...>:
>
>> It doesn't really matter.  Either is fine.  Maybe 255 would have been a
>> more harmonious choice.  Oh well :)
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Dclib-devel mailing list
>> Dcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>>
>>
>