On this video, I continue to discuss the flow of things for starting a Tensorflow training session and getting the training data ready for testing.
I guess in this next section of the workflow we left off in when I was talking about configuring training been the section would come about the execution of the training and what do we’re going to talk about is maybe sound on thoughts that I have about scheduler so but first let’s do this this review the execution portion of of Ed’s Ed’s on his face and steps before starting the execution so it in and also to start treating the tab here but the before start treated tablets read this do she quick review of Ed’s on execution the start of here against the command line part so the going on here in the steps year doing the training but once you did on the configurations and you updated your nine since as such as he describes them here the next step that’s he has been doing is executing this Python script right here now for the present I’m building this application what I want to do is have a series of may be several computers with CPUs listening to the database the record the database and stuff like that see in there’s any most be trained so and again as he says here is ever since self-correcting then issue initializing we should start seeing this right here worth actually executed the steps so on one command line is just a looking at that were Nymex are to larger social is the tents floorboard this of the representation that I don’t think been abusing there is another script that I might add into this and I’ll bring that up they give here and there later going this video and then on once you’re going training and you actually export the inference graph am mortified about little bit about that because I one at is part of this step when training the day set because the next am an expert the training set so executing the training is going to go much into exporting the combined these maybe within the session and then it’s can be leaving to this part of the swim Lane which is the data so the repository have that set of where we have will a web portal and are actually running some tests on against that train data so this this will may be a limitation times so list viscous started here on so we have put here forehand up with here tensor flow training it’s been and are some names here in the closes cancer flow training change this to you so yes in this was his execute training and in that I had another face year and then this is export training so afterward the way training we want to exported this continue on so you have here is for a lack of better word a media server I guess you want to call it this Collett between a media server database discard discard it repository of data so this swim Lane is going to represent our database and are on spirit presented database and the CEU school here down the server in the civic to get a just generic picture of all that what I was looking for you he sure That and this year so this means can the VR acting the server the crisis our database discard it. To be for now I don’t the government bring him to may be using my skill but at the pieces. To be for now for developing and so this swim Lane of talks on it will be aired a repository poster for data now our next something that we can have going ahead shortening here is under you all previous. His videos have this talk about this standalone application assembled that the schedule application or script and unrecorded the scheduler for now Gazette at a given time it’s going the wrong and this is most likely is Amana be creating this in PHP because I’m more familiar with PHP and Caesar for me run it particular run it in any script if you’re following with this with me it Karenina job long or the Karenina Python as well but I’m not a big Python programmer to speed up this process him a run things in’s PHP exec a web application development right so quickly the next one here is the actual tensor flow API so you so the media server here is can it be a repository that is going to represent were for for right now is Simple we have all our images for have all audios primary images trying but it also has all the ideals all the media that we need to process and is 30 for this step but in general right so him and inning with the audio spectrogram is good be in images can be stored in this location where it’s good be available not the scheduler can be said multiple hardware CPUs so in the scheduler can been application I can have flaccid want to three of for CPUs not can go to the cards does get the motherboards as required to run a CPU a CPU but also has a GPU in the back and run this application so Islamicist is in the network and next and be able to ask the media server in the database on the network and it is just to up the database to his is to execute so what to all we want to do is put a starting point and the starting point is going to be right here and is a member little bit so now an Ed’s going to the had here course was manual commandline interface year where he would run whole entire commandline script year for did certain into space so for us what’s can happen it is be are going to provide a status in the database online configuration here where we haven’t him you haven’t going to details yet but here it’s cannot is young basin the changes of the functional user is good is can been an eternity schedule things into the database and then been it it’s can you to do at this point here create all the require files and queue it up for scheduler to run off I might migrate this over to the step only here tents for training because I do one the person is very using this in the kick in the butt data wanted to sit there and check this but anyway’s so that we want to do is actually the one this that and are party described that you my plans as far as that is party going hand in represent that only here in this step a lot easier not better again this this is sexy defining the entire applications of this flowchart might get a little bit big so step number one words you do here is that the scheduler listens to the database every in intervals you see you can insert object work I continues see here will have basic shapes to have something that it’s a circle now see what here in general is looking for Parkinson’s symbols busy a concepts books sorry guys Schedule and and pretty sure is downloading we’re looking for calendar I the civic create one here will but it and then this change this color change the color to something they link tree in the change this to start green the script discussed together shrinkable to one unset look here this morning no no no no no justified in So this continue making it’s want here on ships color to bed tax and him so we haven’t enter will check I want to do this so basically this is can have been every night every second but the same every 10 seconds so for this this this step is can a check every 10 seconds he will be a condition move this over here because it’s can affect be a that so let’s go back to basics pictures us what’s can you said never to decision tree here and actually delete that I’m going here so equated is and then I she had want… And one it to represent the data family and to so what wanted to do is this is and what’s can happen is that this connector here is in a change it to have no in the Queen is just is just sending a query to the database my invest this images that represent that way that it’s this dotted line is just an inquiry get a response back in here is you have your logic same are there any models made for training and would have here is is no than just can come back here and wait every 10 seconds so it is then next step will be on the Eubanks I patient domicile but to hear and is the the next step is to lock the model to this scheduler less than again we are going to at this into the database and this is it’s a step that but we just can it be discussing into the database so the next step is as we want to make sure that it this particular script queries the database and in see the inquiry the response as yet there is a model it’s good is can unlock that model to do script so if I have multiple scripts running in your also check the database and are available for processing models that will collide another was are not asking for the same on right so first gone forever guess that query and there in a goal so you putting some more logic essays did model pair up with scheduler so when Crary again when it when inquiry this in this and it’s it’s it’s pretty much doing to check in on that he did was I successfully to get my quarrying their first right so again I’m I have several servers in one may have quarried the database in their it’s and the prizes of Locke in the database the same time I’m sending my quick to check that is said is just a way of checking to see if I was successful in Locke in the database to this particular script and have I am not proceed to the next step but if I’m not it’s can go over here back into the SKU it’s can await it’s this pretty much can await 10 seconds in his cookware the database again so year on Sonata tomatoes Locke to scheduler what we was life way to do is to start creating all the files and this is the point where I was thinking about adding that’s that he here for training so now we can start well-integrated to there’s was step name and enemy let me backtrack here one this one step year where Ed’s tutorial in and out and I wanted to mention this because we didn’t talk about here in the next step this one step year four he talks about while you’re looking at images you want to have been the mistaking the 4 to 1 ratio in images together images and labels and the guy so here after you have all the pictures you need moved 20% of them to the object detection test directory and 80% of them to the Archduke training directory a sure that there’s millions of pictures in both test and training directory’s so this one sub step of these data collection that happens here what will good do on this this step want to ensure that that happens is that again we can not good do it on to the adhere the database and what we want to do is we want to set sets train terrorists ratio of images for and that ratio is going to be for training to one test so what it these here in a left that authenticity become want to talk about here at this point we can automate for train to one test ratios of all the images that we half for the model when we do the configurations here we want to make sure that we I a step may be here right here web portal the labeling specifications web portal with gallery to verify label date in an adding here and ensure train.test images so when you buy that the same as for me I am on a managing a model myself. No one at this model and him and have thousands and thousands and thousands of images but I wanted to test against select the selected images that I know there should be good to test with I one have the option to manually set that that this series images that I processed I wanted to be in the train bucket in the set of images I wanted to go into the test rocket soul when it goes into the database in a Marquis de Sade so it’s good be a manual set up in while potential full training is said here we want we want to make sure that when the script is Ron that it does not adjust those manual sets in other words on look where the database for list of all the images and him and is saying okay for all the labels amendment party have the script Ponting all the labels and he said okay for the my labels of a I want to get account and I was said before can one racial and and for all the labels be there have the same images as a zoning and that they already and train I want to make sure that those numbers are included this is the labeling goes through right and as as I start doing it, to make sure that is is not necessary the images that I’m going to four point racial bias want to make sure that the ratio is maintained for the labels in in its he doesn’t it is a say that but in my experiences using the method that he’s using he generalizes it here for you just get also be images since images here summons there but and an image images here you see some of images for example you have on the cards you have tens Jackson and what happens if you have 20 jacks images and you only have five with the tens and then you have a mixed batch of these so you do we do the training you want to make sure that you have the best counts of the labels not answer the images because they see you have a bunch of these in an inning here you have no jacks right in the see you have you set aside five of the scene here and that constitute 20% of your test and it will never be able to test against the jacks that makes sense that we can see attended his temperament but anyway this will translate before the one I think is better if you compare them to the labels summoned putting here for the model they went as far to point that so after that is done overseas user the next step year is to start creating files don’t like reading files is to check so you have to to queries here so again these are just quarried green these were the repository the actually can have these two to queries in this step year less at the same of course one his unit query the database for the list of images and then so you see it up you know query for list of images locations from the for the database and check to see if they exists in the media server hosting the body the song want to represent but this is to this so let’s on progress you said is do that soon the recording here and just to represent it’s the quirks to so the database again within of be stored everything his database and or issues you see if things are there and the media server starting all our media on a content write some the next decision year is does image exists so you might ask wall wiring user quarried while because these two are to independence databases repositories again the media server might be a USB thumb drive that city not there and this may be another CPUs separate and we to check that the media server has the actual file in order to process training and is in a dozen exists than will we want to do is tags the database to say will well was that is a this he time now been wanting visual like related there is good his posterior I came back so what we wanted to do is my left can thought here is So we wanted to check we want to check and images there is neither will we do is no set not to use so and images it image does not exist in the media server will what I do is looking to market as set not to use for that particular image so and maybe later on to find out left to them it’s but for this script to run successfully that image must exist and in my be some kind will my have to run some can audit where is actually on being all the images is just is a check that we want to make sure that it can axes and stuff like that so image does exists for we want to do next after this is a say this I guess up at this point we have the model defined have the labels the board once we checked all the images exist this script at this point actually I was at this point in the end that these it is good enough here what I want to do is the MIB sooner case not have percent sure yet so is one of the things that like by video sometimes is not consistent but so here is set up a working directory so in the media server it’s can set up a working directory where it’s can the don’t all the files they can create the log files everything is in to be done to to this directory and I wanted to be Indymedia server where the media server is so after this folder good do is start creating these files that required in them is a might bit is to create these files here will him so he pushes will will will will will will will will continue pushing is out so with this would lead do here is we are going to create the super the CSV file for false train and test your member them get creates two files each of this in this right so the next thing of in is a decision point father created successfully sold and adhere the step log error and then would to the this could be no been a lot of their and it is this this path is can go back to here so this log error here is can go into the database special this year is slated up with the second think so and then this will is America’s it’s not working out us I wanted to ask no wanted to move it up with to do it is there will elected I poops and with do this this can go away and then I wanted to go this way next that that every weapons inspectors back this to at a struggling to represent this so Laguerre on log file in query TV to Mark status there is soul marched as Mark nonmarketable essay of the status as error in said to not ready so close that status will be failed to create CSV file so the next step year of its is successful is to creates generate the TF records party is good be the same logic create do TF records that I script in can have file created successfully and of course the no here in the yes continue on and in the creating the label map.P be GPX file to file created and an apprentice will what in God’s creation this particular right so in one file here that it we he did not discuss and when the things that I wanted to mention was on the previous you were talking about running this faster our CNN conception to pets configuration file at had used in you can get it on the other models that we discussed right but we also needs create this and will to build a web portal to update these attributes and within that we are going to have in the database so with that I want to get this that the tool and to select this to for reference going here and I wanted this here to remind is that this is here this week of the year called is fatter no faster RCC and conception person to so part of our files here is to create that the media this over some can do is creates the Lakota config file and then put decision father created successfully again it’s not been a lot that entry the status in the database and discover back in askew but it is so everything appears to so by this point all the files are created when the do is is a nonstop this video session here and again this this videos that this this whole steps the project thickwalled to… But when on to do this is a nonstop here restarting your session and to continue talking about this flow and spirit back she guys in the next video