Are long sentences not good for deep learning models?












-1















Interested to know if long sentences are good for tensor2tensor model training. And why or why not?






tensorflow tensorflow-datasets tensor2tensor







share|improve this question



























    -1















    Interested to know if long sentences are good for tensor2tensor model training. And why or why not?






    tensorflow tensorflow-datasets tensor2tensor







    share|improve this question

























      -1












      -1








      -1








      Interested to know if long sentences are good for tensor2tensor model training. And why or why not?






      tensorflow tensorflow-datasets tensor2tensor







      share|improve this question














      Interested to know if long sentences are good for tensor2tensor model training. And why or why not?






      tensorflow tensorflow-datasets tensor2tensor




      tensorflow tensorflow-datasets tensor2tensor






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked yesterday









      echan00echan00

      1,1741819




      1,1741819
























          1 Answer
          1






          active

          oldest

          votes


















          0














          Ideally, the training data should have the same distribution of sentence lengths as the target test data. E.g. in machine translation, if long sentences are intended to be translated by the final model, similarly long sentences should be used also for training. The Transformer model seems to not generalize to longer sentences than were used for training, but limiting the maximum sentence length in training allows to use higher batch sizes, which is helpful (Popel and Bojar, 2018).






          share|improve this answer
























          • thanks very useful link

            – echan00
            yesterday











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54251771%2fare-long-sentences-not-good-for-deep-learning-models%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0














          Ideally, the training data should have the same distribution of sentence lengths as the target test data. E.g. in machine translation, if long sentences are intended to be translated by the final model, similarly long sentences should be used also for training. The Transformer model seems to not generalize to longer sentences than were used for training, but limiting the maximum sentence length in training allows to use higher batch sizes, which is helpful (Popel and Bojar, 2018).






          share|improve this answer
























          • thanks very useful link

            – echan00
            yesterday
















          0














          Ideally, the training data should have the same distribution of sentence lengths as the target test data. E.g. in machine translation, if long sentences are intended to be translated by the final model, similarly long sentences should be used also for training. The Transformer model seems to not generalize to longer sentences than were used for training, but limiting the maximum sentence length in training allows to use higher batch sizes, which is helpful (Popel and Bojar, 2018).






          share|improve this answer
























          • thanks very useful link

            – echan00
            yesterday














          0












          0








          0







          Ideally, the training data should have the same distribution of sentence lengths as the target test data. E.g. in machine translation, if long sentences are intended to be translated by the final model, similarly long sentences should be used also for training. The Transformer model seems to not generalize to longer sentences than were used for training, but limiting the maximum sentence length in training allows to use higher batch sizes, which is helpful (Popel and Bojar, 2018).






          share|improve this answer













          Ideally, the training data should have the same distribution of sentence lengths as the target test data. E.g. in machine translation, if long sentences are intended to be translated by the final model, similarly long sentences should be used also for training. The Transformer model seems to not generalize to longer sentences than were used for training, but limiting the maximum sentence length in training allows to use higher batch sizes, which is helpful (Popel and Bojar, 2018).







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered yesterday









          Martin PopelMartin Popel

          2,372620




          2,372620













          • thanks very useful link

            – echan00
            yesterday



















          • thanks very useful link

            – echan00
            yesterday

















          thanks very useful link

          – echan00
          yesterday





          thanks very useful link

          – echan00
          yesterday


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54251771%2fare-long-sentences-not-good-for-deep-learning-models%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Homophylophilia

          Image
          Haec pagina ad tempus protecta est ob maleficia perpetua. Quaesemus, emendationes suggere in pagina disputationis vel magistratum ad hanc deprotegendam pete. Homophylophilia Hyacinthus complexus ab amante suo Apolline, in pictura Mors Hyacinthi quam Jean Broc pinxit. Homophylophilia est sui sexus vel generis amor vel mores romantici vel sexuales, res venerea masculina in masculos, aut mulier in mulieres appetentes. Hoc ingenium sexuale est "exemplar permanens vel inclinatio ad experiendam sexualem, adfectionalem, vel romanticam vim attrahendi," praecipue vel solum ex hominibus eiusdem sexús; notio "etiam ad hominis iudicium identitatis privatae socialisque in illis attractionibus conditum refert, in moribus quoque qui illos exprimant, et in societate communitatis aliorum qui illos communicent." [1] [2] Per aevos alii aliter rem definiverunt, sed Occidentali nostrae aetatis cultu civili non tantum ad ipsa acta erotica spectat, sed etiam igitur
          Read more

          Updating UILabel text programmatically using a function

          Image
          0 so I am trying to update the UIlabel text using a func. I have seen few other questions like mine and I tried their method but it doesn't work. fun returns no value when used in the UIlabel. I have shown my code below.It has to be noted that when I use the function in viewdidload I can retrieve the data. lazy var name: UILabel = { let lb = UILabel() lb.text? = getthename() lb.lineBreakMode = NSLineBreakMode.byWordWrapping lb.textColor = .mainGreen lb.textAlignment = .left lb.font = UIFont.boldSystemFont(ofSize:12.0) return lb }() func getthename() -> String { var wins : String = "" let ud = (Auth.auth().currentUser?.uid) self.ref = Database.database().reference() self.ref.child("Users").child(ud!).observeSingleEvent(of: .value, with: {(snapshot) in
          Read more

          Cloud Functions - OpenCV Videocapture Read method fails for larger files from cloud storage

          Image
          0 I created a cloud function to read a video file from cloud storage and read frames and create an image for each frame. This function works for small files which is 1-2 MB, However, this method fails while reading larger file 130 MB. import cv2 from google.cloud import storage vidcap = cv2.VideoCapture("https://storage.googleapis.com/[bucker_name]/[filename.mp4]") if vidcap.isOpened(): vidcap.open("https://storage.googleapis.com/[bucker_name]/[filename.mp4]") success,image = vidcap.read() print(success) Expected the print will return true and an image will hold an image of a first frame. python-3.x function opencv cloud share | improve this question
          Read more