Most practitioners apply SoftMax on the output to give a normalized probability distribution, as this is in many cases what you'll use a network for - especially in simplified educational material. You'd need to set from_logits to True for the loss function to properly treat the outputs. On the other hand, if your network doesn't apply SoftMax on the output: model = keras.Sequential([ Since this network results in a normalized distribution - when comparing the outputs with target outputs, and grading them via a classification loss function (for the appropriate task) - you should set from_logits to False, or let the default value stay. # Outputs normalized probability - from_logits=False This is also the default value of all loss classes that accept the flag, as most people add an activation='softmax' to their output layers: model = keras.Sequential([ If your network normalizes the output probabilities, your loss function should set from_logits to False, as it's not accepting logits. If your output layer doesn't have a 'softmax' activation, from_logits should be True. If your output layer has a 'softmax' activation, from_logits should be False. Hence, the from_logits flag! When Should from_logits=True? Your loss function has to be informed as to whether it should expect a normalized distribution (output passed through a SoftMax function) or logits. This misconception possibly arises from the short-hand syntax that allows you to add an activation to a layer, seemingly as a single layer, even though it's just shorthand for: output = (n, activation= 'softmax')(x) probabilities: The output is passed through a SoftMax function which normalizes the output into a set of probabilities over n, that all add up to 1.The output of the Dense layer will either return: Depending on the output layer of your network: output = (n, activation= 'softmax')(x)
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |