Ad

How To Remove Consecutive Duplicates In Tensorflow?

For example, input a 1-d tensor:

l_in = [1,1,2,2,3,4,5,5,1,3,5]

I want to remove the consecutive duplicates, which means the output should be:

l_out = [1,2,3,4,5,1,3,5]

However, the tf.unique function only returns unique elements, indicating that the last three elements will also be eliminated. The output of tf.unique is:

[1,2,3,4,5], [0,0,1,1,2,3,4,4,0,2,4] = tf.unique(l_in)

where the second item is the corresponding IDs.

Is there any way to only remove consecutive duplicates while preserving non-duplicated and non-unique elements?

Ad

Answer

For 1D tensors, using array rotation/shifting:

import tensorflow as tf

l_in = tf.constant([1,1,2,2,3,4,5,5,1,3,5])
l_left_shift = tf.concat((l_in[1:], [0]), axis=0)
mask_left_shift = tf.not_equal(l_in - l_left_shift, 0)
mask = tf.concat(([True], mask_left_shift[:-1]), axis=0)
l_out = tf.boolean_mask(l_in, mask)

with tf.Session() as sess:
    print(sess.run(l_out))
# [1 2 3 4 5 1 3 5]

(i.e. the idea is to subtract each element with its right neighbor, then mask out the neighbor if the subtraction result is 0)

Ad
source: stackoverflow.com
Ad