numpy vectorize / more efficient for loop












1















I am performing erosion on an image. The image has been padded accordingly.
In a nutshell, I have a cross element(+) that I put on every pixel of the image and pick the lowest value for that pixel from pixel above, below, right, left and itself.



It is inefficient and I can't figure out a vectorized version. It must be possible since all calculations are done independently of each other.



for y in range(t,image.shape[0]-b):
for x in range(l,image.shape[1]-r):
a1 = numpy.copy(str_ele)
for filter_y in range(a1.shape[0]):
for filter_x in range(a1.shape[1]):
if (not (numpy.isnan(a1[filter_y][filter_x]))):
a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
eroded_image[y][x] = numpy.nanmin(a1)


basically:



Every pixel in final image = min(pixel, above, below, left, right) from the original image



 for y in range(len(eroded_image)):
for x in range(len(eroded_image[1])):
eroded_image2[y][x] = numpy.nanmin(str_ele*image2[y:y+len(str_ele),x:x+(len(str_ele[1]))])


This is what I now have. Still 2 loops.










share|improve this question





























    1















    I am performing erosion on an image. The image has been padded accordingly.
    In a nutshell, I have a cross element(+) that I put on every pixel of the image and pick the lowest value for that pixel from pixel above, below, right, left and itself.



    It is inefficient and I can't figure out a vectorized version. It must be possible since all calculations are done independently of each other.



    for y in range(t,image.shape[0]-b):
    for x in range(l,image.shape[1]-r):
    a1 = numpy.copy(str_ele)
    for filter_y in range(a1.shape[0]):
    for filter_x in range(a1.shape[1]):
    if (not (numpy.isnan(a1[filter_y][filter_x]))):
    a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
    eroded_image[y][x] = numpy.nanmin(a1)


    basically:



    Every pixel in final image = min(pixel, above, below, left, right) from the original image



     for y in range(len(eroded_image)):
    for x in range(len(eroded_image[1])):
    eroded_image2[y][x] = numpy.nanmin(str_ele*image2[y:y+len(str_ele),x:x+(len(str_ele[1]))])


    This is what I now have. Still 2 loops.










    share|improve this question



























      1












      1








      1


      1






      I am performing erosion on an image. The image has been padded accordingly.
      In a nutshell, I have a cross element(+) that I put on every pixel of the image and pick the lowest value for that pixel from pixel above, below, right, left and itself.



      It is inefficient and I can't figure out a vectorized version. It must be possible since all calculations are done independently of each other.



      for y in range(t,image.shape[0]-b):
      for x in range(l,image.shape[1]-r):
      a1 = numpy.copy(str_ele)
      for filter_y in range(a1.shape[0]):
      for filter_x in range(a1.shape[1]):
      if (not (numpy.isnan(a1[filter_y][filter_x]))):
      a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
      eroded_image[y][x] = numpy.nanmin(a1)


      basically:



      Every pixel in final image = min(pixel, above, below, left, right) from the original image



       for y in range(len(eroded_image)):
      for x in range(len(eroded_image[1])):
      eroded_image2[y][x] = numpy.nanmin(str_ele*image2[y:y+len(str_ele),x:x+(len(str_ele[1]))])


      This is what I now have. Still 2 loops.










      share|improve this question
















      I am performing erosion on an image. The image has been padded accordingly.
      In a nutshell, I have a cross element(+) that I put on every pixel of the image and pick the lowest value for that pixel from pixel above, below, right, left and itself.



      It is inefficient and I can't figure out a vectorized version. It must be possible since all calculations are done independently of each other.



      for y in range(t,image.shape[0]-b):
      for x in range(l,image.shape[1]-r):
      a1 = numpy.copy(str_ele)
      for filter_y in range(a1.shape[0]):
      for filter_x in range(a1.shape[1]):
      if (not (numpy.isnan(a1[filter_y][filter_x]))):
      a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
      eroded_image[y][x] = numpy.nanmin(a1)


      basically:



      Every pixel in final image = min(pixel, above, below, left, right) from the original image



       for y in range(len(eroded_image)):
      for x in range(len(eroded_image[1])):
      eroded_image2[y][x] = numpy.nanmin(str_ele*image2[y:y+len(str_ele),x:x+(len(str_ele[1]))])


      This is what I now have. Still 2 loops.







      python numpy image-processing vectorization image-morphology






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Dec 10 '18 at 17:20









      Cris Luengo

      20.2k52249




      20.2k52249










      asked Nov 25 '18 at 21:34









      GaryGary

      83




      83
























          1 Answer
          1






          active

          oldest

          votes


















          2














          If image is a NaN-padded array, and you are eroding with a cross-shaped footprint,
          you could remove the for-loops by stacking slices of image (to effectively shift the image up, left, right, and down)
          and then apply np.nanmin to the stack of slices.



          import numpy as np

          def orig(image):
          t, l, b, r = 1, 1, 1, 1
          str_ele = np.array([[np.nan, 1, np.nan], [1, 1, 1], [np.nan, 1, np.nan]], dtype='float')
          str_ele_center_x, str_ele_center_y = 1, 1
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          for y in range(t,image.shape[0]-b):
          for x in range(l,image.shape[1]-r):
          a1 = np.copy(str_ele)
          for filter_y in range(a1.shape[0]):
          for filter_x in range(a1.shape[1]):
          if (not (np.isnan(a1[filter_y][filter_x]))):
          a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
          eroded_image[y][x] = np.nanmin(a1)
          return eroded_image

          def erode(image):
          result = np.stack([image[1:-1, 1:-1], image[2:, 1:-1], image[:-2, 1:-1], image[1:-1, 2:], image[1:-1, :-2]])
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          eroded_image[1:-1, 1:-1] = np.nanmin(result, axis=0)
          return eroded_image

          image = np.arange(24).reshape(4,6)
          image = np.pad(image.astype(float), 1, mode='constant', constant_values=np.nan)


          yields



          In [228]: erode(image)
          Out[228]:
          array([[nan, nan, nan, nan, nan, nan, nan, nan],
          [nan, 0., 0., 1., 2., 3., 4., nan],
          [nan, 0., 1., 2., 3., 4., 5., nan],
          [nan, 6., 7., 8., 9., 10., 11., nan],
          [nan, 12., 13., 14., 15., 16., 17., nan],
          [nan, nan, nan, nan, nan, nan, nan, nan]])




          For the small example image above, erode appears to be around 33x faster than orig:



          In [23]: %timeit erode(image)
          10000 loops, best of 3: 35.6 µs per loop

          In [24]: %timeit orig(image)
          1000 loops, best of 3: 1.19 ms per loop

          In [25]: 1190/35.6
          Out[25]: 33.42696629213483





          share|improve this answer





















          • 1





            For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

            – Paul Panzer
            Nov 26 '18 at 0:13











          • @PaulPanzer: Thanks very much for the improvement.

            – unutbu
            Nov 26 '18 at 6:33











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53472236%2fnumpy-vectorize-more-efficient-for-loop%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          2














          If image is a NaN-padded array, and you are eroding with a cross-shaped footprint,
          you could remove the for-loops by stacking slices of image (to effectively shift the image up, left, right, and down)
          and then apply np.nanmin to the stack of slices.



          import numpy as np

          def orig(image):
          t, l, b, r = 1, 1, 1, 1
          str_ele = np.array([[np.nan, 1, np.nan], [1, 1, 1], [np.nan, 1, np.nan]], dtype='float')
          str_ele_center_x, str_ele_center_y = 1, 1
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          for y in range(t,image.shape[0]-b):
          for x in range(l,image.shape[1]-r):
          a1 = np.copy(str_ele)
          for filter_y in range(a1.shape[0]):
          for filter_x in range(a1.shape[1]):
          if (not (np.isnan(a1[filter_y][filter_x]))):
          a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
          eroded_image[y][x] = np.nanmin(a1)
          return eroded_image

          def erode(image):
          result = np.stack([image[1:-1, 1:-1], image[2:, 1:-1], image[:-2, 1:-1], image[1:-1, 2:], image[1:-1, :-2]])
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          eroded_image[1:-1, 1:-1] = np.nanmin(result, axis=0)
          return eroded_image

          image = np.arange(24).reshape(4,6)
          image = np.pad(image.astype(float), 1, mode='constant', constant_values=np.nan)


          yields



          In [228]: erode(image)
          Out[228]:
          array([[nan, nan, nan, nan, nan, nan, nan, nan],
          [nan, 0., 0., 1., 2., 3., 4., nan],
          [nan, 0., 1., 2., 3., 4., 5., nan],
          [nan, 6., 7., 8., 9., 10., 11., nan],
          [nan, 12., 13., 14., 15., 16., 17., nan],
          [nan, nan, nan, nan, nan, nan, nan, nan]])




          For the small example image above, erode appears to be around 33x faster than orig:



          In [23]: %timeit erode(image)
          10000 loops, best of 3: 35.6 µs per loop

          In [24]: %timeit orig(image)
          1000 loops, best of 3: 1.19 ms per loop

          In [25]: 1190/35.6
          Out[25]: 33.42696629213483





          share|improve this answer





















          • 1





            For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

            – Paul Panzer
            Nov 26 '18 at 0:13











          • @PaulPanzer: Thanks very much for the improvement.

            – unutbu
            Nov 26 '18 at 6:33
















          2














          If image is a NaN-padded array, and you are eroding with a cross-shaped footprint,
          you could remove the for-loops by stacking slices of image (to effectively shift the image up, left, right, and down)
          and then apply np.nanmin to the stack of slices.



          import numpy as np

          def orig(image):
          t, l, b, r = 1, 1, 1, 1
          str_ele = np.array([[np.nan, 1, np.nan], [1, 1, 1], [np.nan, 1, np.nan]], dtype='float')
          str_ele_center_x, str_ele_center_y = 1, 1
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          for y in range(t,image.shape[0]-b):
          for x in range(l,image.shape[1]-r):
          a1 = np.copy(str_ele)
          for filter_y in range(a1.shape[0]):
          for filter_x in range(a1.shape[1]):
          if (not (np.isnan(a1[filter_y][filter_x]))):
          a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
          eroded_image[y][x] = np.nanmin(a1)
          return eroded_image

          def erode(image):
          result = np.stack([image[1:-1, 1:-1], image[2:, 1:-1], image[:-2, 1:-1], image[1:-1, 2:], image[1:-1, :-2]])
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          eroded_image[1:-1, 1:-1] = np.nanmin(result, axis=0)
          return eroded_image

          image = np.arange(24).reshape(4,6)
          image = np.pad(image.astype(float), 1, mode='constant', constant_values=np.nan)


          yields



          In [228]: erode(image)
          Out[228]:
          array([[nan, nan, nan, nan, nan, nan, nan, nan],
          [nan, 0., 0., 1., 2., 3., 4., nan],
          [nan, 0., 1., 2., 3., 4., 5., nan],
          [nan, 6., 7., 8., 9., 10., 11., nan],
          [nan, 12., 13., 14., 15., 16., 17., nan],
          [nan, nan, nan, nan, nan, nan, nan, nan]])




          For the small example image above, erode appears to be around 33x faster than orig:



          In [23]: %timeit erode(image)
          10000 loops, best of 3: 35.6 µs per loop

          In [24]: %timeit orig(image)
          1000 loops, best of 3: 1.19 ms per loop

          In [25]: 1190/35.6
          Out[25]: 33.42696629213483





          share|improve this answer





















          • 1





            For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

            – Paul Panzer
            Nov 26 '18 at 0:13











          • @PaulPanzer: Thanks very much for the improvement.

            – unutbu
            Nov 26 '18 at 6:33














          2












          2








          2







          If image is a NaN-padded array, and you are eroding with a cross-shaped footprint,
          you could remove the for-loops by stacking slices of image (to effectively shift the image up, left, right, and down)
          and then apply np.nanmin to the stack of slices.



          import numpy as np

          def orig(image):
          t, l, b, r = 1, 1, 1, 1
          str_ele = np.array([[np.nan, 1, np.nan], [1, 1, 1], [np.nan, 1, np.nan]], dtype='float')
          str_ele_center_x, str_ele_center_y = 1, 1
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          for y in range(t,image.shape[0]-b):
          for x in range(l,image.shape[1]-r):
          a1 = np.copy(str_ele)
          for filter_y in range(a1.shape[0]):
          for filter_x in range(a1.shape[1]):
          if (not (np.isnan(a1[filter_y][filter_x]))):
          a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
          eroded_image[y][x] = np.nanmin(a1)
          return eroded_image

          def erode(image):
          result = np.stack([image[1:-1, 1:-1], image[2:, 1:-1], image[:-2, 1:-1], image[1:-1, 2:], image[1:-1, :-2]])
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          eroded_image[1:-1, 1:-1] = np.nanmin(result, axis=0)
          return eroded_image

          image = np.arange(24).reshape(4,6)
          image = np.pad(image.astype(float), 1, mode='constant', constant_values=np.nan)


          yields



          In [228]: erode(image)
          Out[228]:
          array([[nan, nan, nan, nan, nan, nan, nan, nan],
          [nan, 0., 0., 1., 2., 3., 4., nan],
          [nan, 0., 1., 2., 3., 4., 5., nan],
          [nan, 6., 7., 8., 9., 10., 11., nan],
          [nan, 12., 13., 14., 15., 16., 17., nan],
          [nan, nan, nan, nan, nan, nan, nan, nan]])




          For the small example image above, erode appears to be around 33x faster than orig:



          In [23]: %timeit erode(image)
          10000 loops, best of 3: 35.6 µs per loop

          In [24]: %timeit orig(image)
          1000 loops, best of 3: 1.19 ms per loop

          In [25]: 1190/35.6
          Out[25]: 33.42696629213483





          share|improve this answer















          If image is a NaN-padded array, and you are eroding with a cross-shaped footprint,
          you could remove the for-loops by stacking slices of image (to effectively shift the image up, left, right, and down)
          and then apply np.nanmin to the stack of slices.



          import numpy as np

          def orig(image):
          t, l, b, r = 1, 1, 1, 1
          str_ele = np.array([[np.nan, 1, np.nan], [1, 1, 1], [np.nan, 1, np.nan]], dtype='float')
          str_ele_center_x, str_ele_center_y = 1, 1
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          for y in range(t,image.shape[0]-b):
          for x in range(l,image.shape[1]-r):
          a1 = np.copy(str_ele)
          for filter_y in range(a1.shape[0]):
          for filter_x in range(a1.shape[1]):
          if (not (np.isnan(a1[filter_y][filter_x]))):
          a1[filter_y][filter_x] = str_ele[filter_y][filter_x]*image[y+(filter_y-str_ele_center_y)][x+(filter_x-str_ele_center_x)]
          eroded_image[y][x] = np.nanmin(a1)
          return eroded_image

          def erode(image):
          result = np.stack([image[1:-1, 1:-1], image[2:, 1:-1], image[:-2, 1:-1], image[1:-1, 2:], image[1:-1, :-2]])
          eroded_image = np.full_like(image, dtype='float', fill_value=np.nan)
          eroded_image[1:-1, 1:-1] = np.nanmin(result, axis=0)
          return eroded_image

          image = np.arange(24).reshape(4,6)
          image = np.pad(image.astype(float), 1, mode='constant', constant_values=np.nan)


          yields



          In [228]: erode(image)
          Out[228]:
          array([[nan, nan, nan, nan, nan, nan, nan, nan],
          [nan, 0., 0., 1., 2., 3., 4., nan],
          [nan, 0., 1., 2., 3., 4., 5., nan],
          [nan, 6., 7., 8., 9., 10., 11., nan],
          [nan, 12., 13., 14., 15., 16., 17., nan],
          [nan, nan, nan, nan, nan, nan, nan, nan]])




          For the small example image above, erode appears to be around 33x faster than orig:



          In [23]: %timeit erode(image)
          10000 loops, best of 3: 35.6 µs per loop

          In [24]: %timeit orig(image)
          1000 loops, best of 3: 1.19 ms per loop

          In [25]: 1190/35.6
          Out[25]: 33.42696629213483






          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 26 '18 at 6:33

























          answered Nov 25 '18 at 23:00









          unutbuunutbu

          549k10111781237




          549k10111781237








          • 1





            For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

            – Paul Panzer
            Nov 26 '18 at 0:13











          • @PaulPanzer: Thanks very much for the improvement.

            – unutbu
            Nov 26 '18 at 6:33














          • 1





            For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

            – Paul Panzer
            Nov 26 '18 at 0:13











          • @PaulPanzer: Thanks very much for the improvement.

            – unutbu
            Nov 26 '18 at 6:33








          1




          1





          For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

          – Paul Panzer
          Nov 26 '18 at 0:13





          For a proper image (scipy.misc.face().mean(2) I get a more than twofold speedup by stacking (and taking the nanmin) along axis 0 instead of -1.

          – Paul Panzer
          Nov 26 '18 at 0:13













          @PaulPanzer: Thanks very much for the improvement.

          – unutbu
          Nov 26 '18 at 6:33





          @PaulPanzer: Thanks very much for the improvement.

          – unutbu
          Nov 26 '18 at 6:33


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53472236%2fnumpy-vectorize-more-efficient-for-loop%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

          Calculate evaluation metrics using cross_val_predict sklearn

          Insert data from modal to MySQL (multiple modal on website)