dclib-devel — A list for developers to discuss issues

Re: [Dclib-devel] [dclib-devel] Implementing a GAN with dlib

[Eloi Du B. <elo...@gm...>]

Yes I did; shall I call the backward function on each of the layers or will
the function do this for me?

Thanks,

2017-10-22 7:04 GMT-05:00 Davis King <dav...@gm...>:

> Did you read the documentation for back_propagate_error()? See here
> http://dlib.net/dlib/dnn/core_abstract.h.html#add_layer.  It answers this
> question in great detail. And yes, the gradients are all available.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] [dclib-devel] Implementing a GAN with dlib

[Davis K. <dav...@gm...>]

Did you read the documentation for back_propagate_error()? See here
http://dlib.net/dlib/dnn/core_abstract.h.html#add_layer.  It answers this
question in great detail. And yes, the gradients are all available.

Re: [Dclib-devel] [dclib-devel] Implementing a GAN with dlib

[Eloi Du B. <elo...@gm...>]

Ok,

I'm starting to dig a little more to understand how dlib is made, I think I
have an idea on how to implement this. I have few questions about forward
vs backpropagation:

I'm making a discriminator defined as:
// A 1 filter; 5x1 conv layer that does 2x downsampling
template <typename SUBNET>
using con2 = dlib::con<1, 1, 5, 1, 2, SUBNET>;

template<class SUBNET>
using DiscriminatorT =
dlib::loss_binary_log<dlib::fc<1,dlib::relu<dlib::max_pool<1, 2, 1, 2,
dlib::relu<dlib::max_pool<1, 2, 1, 2, dlib::relu<con2<SUBNET>>>>>>>>;

This just to detect either a vector of 128 floats follows a gaussian
distribution or not.

Then, my discriminator is trained over some test data, positive and
negative examples (gaussian vs noise).

Now, I'm trying to check if the forward function on the net is giving the
same result as a call to the neural net,

        std::vector<float> classificationForTest =
discriminator(testVectors);

        resizable_tensor tempTensor;
        discriminator.to_tensor(&testVectors[0], &testVectors[0]+1,
tempTensor);

        auto & tensorOut = discriminator.subnet().forward(tempTensor);

assert( *(float*)tensorOut.host() == classificationForTest[0]);

This is test passes, the forward pass is what I thought it was.

Now, I'm trying to do a back propagation of the error.
So let's say classificationForTest[0] is -8 and it should be 8, my error is
16.
I'm tempted to do something like (pseudo code):
        discriminator.subnet().back_propagate_error(tensor = 16);

My question is:

How to back-propagate and rewind the full network to get the error
introduced by each of my scalars of an input vector applied to the
discriminator?
If I get this, then I know how to solve my GAN as I would inject this error
into the output tensor of the generator.

Many thanks for any help,


2017-10-21 20:08 GMT-05:00 Davis King <dav...@gm...>:

> I would make a network that forks into two subnetworks and then write my
> own loss layer that did whatever you want to do.  I doubt you could
> implement any of the GAN papers using a binary classification loss like
> loss_binary_log.  They all have very specific loss functions that only make
> sense in the context of GAN.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] [dclib-devel] Implementing a GAN with dlib

[Davis K. <dav...@gm...>]

I would make a network that forks into two subnetworks and then write my
own loss layer that did whatever you want to do.  I doubt you could
implement any of the GAN papers using a binary classification loss like
loss_binary_log.  They all have very specific loss functions that only make
sense in the context of GAN.

[Dclib-devel] [dclib-devel] Implementing a GAN with dlib

[Eloi Du B. <elo...@gm...>]

Hi,

I would like some advise on how to implement a Generative Adversial Network
with dlib.
I read a lot about this network, and from what I see, I think I need to
make one network containing the Generator and the Discriminator in one net,
concatenated.

Noise -> G -> output sample -> D -> [true, fake]

The thing where I'm stuck on is , when using dlib, how can I do a separate
training on both subnets? How do I extract the output generated sample in
the middle of both?
What loss function do you think I need? I was thinking of loss_binary_log.

Or maybe I'm misunderstanding the principle. Do you have any code example
that someone did with dlib?

Many thanks for any answer,
Bests,
Eloi

Re: [Dclib-devel] FFT for non power2 image dimensions

[Davis K. <dav...@gm...>]

Sweet.

Re: [Dclib-devel] FFT for non power2 image dimensions

[Eloi Du B. <elo...@gm...>]

Ok, I will see what I can do.

2017-08-29 14:32 GMT-05:00 Davis King <dav...@gm...>:

> Yeah, it would be nice.  You should submit a PR for a non power of 2 FFT :)
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] FFT for non power2 image dimensions

[Davis K. <dav...@gm...>]

Yeah, it would be nice.  You should submit a PR for a non power of 2 FFT :)

Re: [Dclib-devel] Reference to a temporary variable

[Davis K. <dav...@gm...>]

No, that's valid C++.

[Dclib-devel] Reference to a temporary variable

[Eloi Du B. <elo...@gm...>]

Just for your info,
I see a mistake here:  spatial_filtering.h +1241
            const matrix<ptype,0,1>& filt =
create_gaussian_filter<ptype>(sigma, max_size);

This a reference to a temporary variable.

Re: [Dclib-devel] Which search strategy should I use for a function with thousands variables

[Davis K. <dav...@gm...>]

Use the LBFGS strategy.

[Dclib-devel] Which search strategy should I use for a function with thousands variables

[Yixun L. <yx...@gm...>]

Hi,
I would like to minimize a single value function with thousands variables.
I can get the derivatives, but it is time consuming.



Thanks.

Yixun

Re: [Dclib-devel] FFT for non power2 image dimensions

[Eloi Du B. <elo...@gm...>]

Ok, thanks,

It would be nice to have it as a default, plus if I understand well, there
is an optimized way to do this:
https://math.stackexchange.com/questions/77118/non-power-of-2-ffts


2017-08-29 11:10 GMT-05:00 Davis King <dav...@gm...>:

> You have to pad your image with zeros so that it is a power of two.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

Re: [Dclib-devel] FFT for non power2 image dimensions

[Davis K. <dav...@gm...>]

You have to pad your image with zeros so that it is a power of two.

[Dclib-devel] FFT for non power2 image dimensions

[Eloi Du B. <elo...@gm...>]

Hi,

I see that there is an FFT implementation, but the problem I have is that
it is expecting a power of 2 length. I'm surprised, is there a way to use
this for non power-2 image dimensions?

Thanks!

Re: [Dclib-devel] CNN for Text Classification

[Davis K. <dav...@us...>]

No, that doesn't make sense.  All these functions are documented, including
join_rows.  If you read the documentation you can see what it does.
http://dlib.net/faq.html#Whereisthedocumentationforobjectfunction

[Dclib-devel] CNN for Text Classification

[Stefan S. <st...@sc...>]

Hi dlib-users,

I guess this question is both related to MITIE and dlib: I would really
like to contribute an example for text classification with a CNN. I'm
working with the MNIST example code and I've got a few questions:

In MITIE I can use pretrained word embeddings from the english model
(total_word_feature_extractor.dat). E.g. the wordrep tool shows the
feature vector for a word with --test <Word>.

What is the correct way to the get a sentence representation? At the
moment I' using the following code:

matrix<float,0,1> sentence_matrix;

for (auto &word : sentence) {
  matrix<float,0,1> feats;
  fe.get_feature_vector(word,feats);

  join_rows(sentence_matrix, feats);
}
testing_set.emplace_back(sentence_matrix);
testing_labels.emplace_back(1);

Of course I have to pad the sentence, but would this code create a
correct sentence representation which I could later use to train the
network? The intention is to create a n x k representation for the
sentence (n = length of sentence, k is length of feature vector from
word embedding). For the concatenation of each word I found the
join_rows method, or should I use something else?


Thanks in advance + regards,

Stefan

Re: [Dclib-devel] new to dlib

[Davis K. <dav...@us...>]

I would get some of those books mentioned on the web page and read them in
the order suggested.  It somewhat depends on how much you want to learn.

As for code development, I use vim, cmake, and a bash shell.

Re: [Dclib-devel] new to dlib

[Ram K. <ra...@gm...>]

just updated the questions. Thanks

1) I like to know which module or area is good for begniners to start?
2) which IDE or tools that is being used to step through the code and debug ?  I am using mac os.

Regards,
Ram

> On Dec 2, 2016, at 5:07 PM, Ram Krishnamurthappa <ra...@gm...> wrote:
> 
> Hi,
> 
> I am new to dlib. I downloaded the code and compiled it and ran test. I started to go through books mentioned in the web page. Please help me in the below questions.
> 
> 1) I like to know which module or area is good for begniners to start?
> 2) which IDE or tools that is being used to step through the code and debug ?
> 
> Thanks for your help
> 
> Regards,
> Ram

[Dclib-devel] new to dlib

[Ram K. <ra...@gm...>]

Hi,

I am new to dlib. I downloaded the code and compiled it and ran test. I started to go through books mentioned in the web page. Please help me in the below questions.

1) I like to know which module or area is good for begniners to start?
2) which IDE or tools that is being used to step through the code and debug ?

Thanks for your help

Regards,
Ram

Re: [Dclib-devel] dlib fatal error with output_tensor

[Davis K. <dav...@us...>]

Don't spam questions to multiple forums.  I answered this on stackoverflow
already.  Moreover, if you had read the documentation for the things you
are using you would have understood why it was wrong in the first place.

On Fri, Dec 2, 2016 at 11:01 AM, <dav...@gm...> wrote:

> Hi everybody,
>
> I am currently working on a face detection application using RCNN
> algorithm. The code in developed in dlib-19.2 C++. My Operating System is
> Windows 10 Home Edition. The framework is Qt 5.7.
>
> My 781 images (height:228 and width:307) inside are inside a vector called
> "images"
>
> *std::vector<matrix<rgb_pixel>> images;*
>
> Afterwards, I split "images" into two vectors: One containing the training
> images (389 in total) and the testing images (391 in total):
>
> *std**:**:vector**<**dlib**:**:matrix**<**dlib**:**:rgb_pixel**>>** training_images(images.begin(), images.begin() + half_size);*
>
> *std**:**:vector**<**dlib**:**:matrix**<**dlib**:**:rgb_pixel**>>** testing_images(images.begin() + half_size, images.end());*
>
> Then, I tried to train the RCNN network using the train() function:
>
> *template <long num_filters, typename SUBNET> using con5d = con<num_filters,5,5,2,2,SUBNET>;*
>
>
>
> *template <long num_filters, typename SUBNET> using con3  = con<num_filters,3,3,1,1,SUBNET>;*
>
>
>
> *template <typename SUBNET> using downsampler  = relu<bn_con<con5d<32, relu<bn_con<con5d<32, relu<bn_con<con5d<32,SUBNET>>>>>>>>>;*
>
>
>
> *template <typename SUBNET> using rcon3  = relu<bn_con<con3<32,SUBNET>>>;*
>
>
>
> *using net_type  = loss_binary_log<con<1,6,6,1,1,rcon3<rcon3<rcon3<downsampler<input_rgb_image_pyramid<pyramid_down<6>>>>>>>>;*
>
>
>
> *net_type net;*
>
> *dnn_trainer<net_type> trainer(net);*
>
> *trainer.set_learning_rate(0.01);*
>
> *trainer.set_min_learning_rate(0.00001);*
>
> *trainer.set_mini_batch_size(5);*
>
> *trainer.be_verbose();*
>
> *trainer.train(training_images, training_labels);*
>
> However, this does not work. The following error appears every single time
> I run the code:
>
> *Error** detected at line 179. Error detected in file ../../Desktop/dlib-19.2/dlib/dnn/loss.h. Error detected in function double dlib::loss_binary_log_::compute_loss_value_and_gradient(const dlib::tensor&, const_label_iterator, SUBNET&) const [with const_label_iterator = __gnu_cxx::__normal_iterator >; SUBNET = dlib::dimpl::subnet_wrapper, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::input_rgb_image_pyramid >, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, true, void>].*
>
>
>
> *Failing** expression was output_tensor.nr() == 1 && output_tensor.nc() == 1 && output_tensor.k() == 1.*
>
> I really do not know what is going on. Could you help me please ?
>
> Thank you very much for all your answers.
>
> David
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Dclib-devel mailing list
> Dcl...@li...
> https://lists.sourceforge.net/lists/listinfo/dclib-devel
>
>

[Dclib-devel] dlib fatal error with output_tensor

[<dav...@gm...>]

Hi everybody,
I am currently working on a face detection application using RCNN algorithm. The code in developed in dlib-19.2 C++. My Operating System is Windows 10 Home Edition. The framework is Qt 5.7.
My 781 images (height:228 and width:307) inside are inside a vector called "images"
std::vector<matrix<rgb_pixel>> images;
Afterwards, I split "images" into two vectors: One containing the training images (389 in total) and the testing images (391 in total):
std::vector<dlib::matrix<dlib::rgb_pixel>> training_images(images.begin(), images.begin() + half_size);
std::vector<dlib::matrix<dlib::rgb_pixel>> testing_images(images.begin() + half_size, images.end());
Then, I tried to train the RCNN network using the train() function:
template <long num_filters, typename SUBNET> using con5d = con<num_filters,5,5,2,2,SUBNET>;

template <long num_filters, typename SUBNET> using con3  = con<num_filters,3,3,1,1,SUBNET>;

template <typename SUBNET> using downsampler  = relu<bn_con<con5d<32, relu<bn_con<con5d<32, relu<bn_con<con5d<32,SUBNET>>>>>>>>>;

template <typename SUBNET> using rcon3  = relu<bn_con<con3<32,SUBNET>>>;

using net_type  = loss_binary_log<con<1,6,6,1,1,rcon3<rcon3<rcon3<downsampler<input_rgb_image_pyramid<pyramid_down<6>>>>>>>>;

net_type net;
dnn_trainer<net_type> trainer(net);
trainer.set_learning_rate(0.01);
trainer.set_min_learning_rate(0.00001);
trainer.set_mini_batch_size(5);
trainer.be_verbose();
trainer.train(training_images, training_labels);
However, this does not work. The following error appears every single time I run the code:
Error detected at line 179. Error detected in file ../../Desktop/dlib-19.2/dlib/dnn/loss.h. Error detected in function double dlib::loss_binary_log_::compute_loss_value_and_gradient(const dlib::tensor&, const_label_iterator, SUBNET&) const [with const_label_iterator = __gnu_cxx::__normal_iterator >; SUBNET = dlib::dimpl::subnet_wrapper, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::add_layer, dlib::input_rgb_image_pyramid >, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, void>, true, void>].

Failing expression was output_tensor.nr() == 1 && output_tensor.nc() == 1 && output_tensor.k() == 1.
I really do not know what is going on. Could you help me please ?
Thank you very much for all your answers.
David

[Dclib-devel] Slicing run-time sized column or row vectors in dlib

[Yogesh K. <yog...@ya...>]

<div>Hi Guys,</div><div>I am beginner in both C++ and Dlib so bear with me.</div><div>When I tried to run some of the examples in my context,</div><div>I have a difficulty in a very simple thing:</div><div><pre><span>typedef</span><span> </span><span>matrix</span>&lt;<span>double</span>,<span>0</span>,<span>1</span>&gt;<span> </span><span>sample_type</span>;</pre><pre>//Samples is a dynamic array generated run time</pre><div> s<span>td</span><span>::</span><span>vector</span><span>&lt;</span><span>sample_type</span><span>&gt;</span><span> </span><span>samples;</span></div> </div><div>what I want to do is to trim the array (extract the subarray)</div><div>like </div><div>s<span>td</span><span>::</span><span>vector</span><span>&lt;</span><span>sample_type</span><span>&gt;</span><span> </span><span>A;</span></div><div><span>A=subm(samples,range(0,10),range(0,10));</span></div><div><span>but it throws the error "No matching function for call to.."</span></div><div> </div><div><span>Thanks a lot in advance.</span></div><div> </div><div><span>Warm Regards</span></div><div><span>Yogesh</span></div>

[Dclib-devel] edgelist graph to a serialized Bayesian Network

[S. A. <sa...@ya...>]

Is there a way to covert a basic edgelist graph to a serialized Bayesian Network?
(should questions be posted on the github issues project)

Re: [Dclib-devel] Multiclass classification

[Davis K. <dav...@us...>]

Yes, you could try that one as well and it might be faster, depending on
the value of N.