How to assign certain samples as negative samples when using sampled_softmax

The API of sampled_softmax_loss goes like:

tf.nn.sampled_softmax_loss(
    weights, 
    biases, 
    labels, 
    inputs, 
    num_sampled, 
    num_classes, 
    num_true=1,
    sampled_values=None, 
    ...
)

I've noticed that arg sampled_values is the one which determines what negatives samples we take and it's returned by a _candidate_sampler function like tf.random.fixed_unigram_candidate_sampler.

And in tf.random.fixed_unigram_candidate_sampler we can decide the probability of each sample chosen as negative sample.

But my question is, how to assign certain sample as negative sample on purpose?

For instance, in the case of recommender system, I'd like to add some hard negative sample to the model. So I want the hard negative samples been chosen for sure, not by probability like in _candidate_sampler function

CodePudding user response：

One possible way to achieve this is to use a custom _candidate_sampler function that always includes the desired hard negative samples, and assigns a high probability to them. For example:

# Define the hard negative samples
hard_negatives = [4, 5, 6]

# Define the custom candidate sampler
def custom_candidate_sampler(num_true, true_classes, num_sampled, unique, range_max, seed=None, name=None):
  # Include the hard negatives in the samples
  samples = hard_negatives   [0, 1, 2, 3]
  # Assign a high probability to the hard negatives
  weights = [0.9] * len(hard_negatives)   [0.1] * (len(samples) - len(hard_negatives))
  return samples, weights

# Use the custom candidate sampler in the sampled_softmax_loss function
sampled_softmax_loss(
  weights, 
  biases, 
  labels, 
  inputs, 
  num_sampled, 
  num_classes, 
  num_true=1,
  sampled_values=custom_candidate_sampler, 
  ...
)

This way, the hard negative samples will always be included in the negative samples used by the sampled_softmax_loss function, and will have a high probability of being chosen. You can adjust the probabilities assigned to the hard negatives and other samples as needed.

CodePudding user response：

you need to understand that the sampler candidates function is only a remarks function and your question is right about how to create a negative sampler.

You don't need to create a negative sampler when you assigned a unique, the sampler is (sampled_candidates, true_expected_count, sampled_expected_count) hard negative is when you add contrast values to significant the candidates. In this way, you can have it with distributions.