用LSTM实现文本生成和生成对抗网络(GAN)

    corpus = """
    The atmosphere in Africa is becoming more and more volatile as a result of deforestation due to intense logging practices. This has led to the extinction of many species, including some rare birds like bushchat, pied bull, cockatoos, and herons that were previously abundant in this region. Some reptiles have also been severely impacted, with crocodiles being particularly vulnerable. The increasing population density and loss of habitats make it difficult for surviving individuals to adapt or develop suitable foraging behaviors or consequently suffer from malnutrition and other diseases. Human activity such as mining, oil palm plantations, fishing boats, and construction projects are also contributing to the spread of disease. In particular, the Bantu languages, which are spoken across West Africa, are experiencing severe hunger and malnutrition caused by rapid population growth and trade.
    """
    
    # Define the mapping of characters to integers
    char2idx = {u:i for i, u in enumerate(set(corpus))}
    idx2char = np.array(list(char2idx.keys()))
    vocab_size = len(char2idx)
    
    text_as_int = np.array([char2idx[c] for c in corpus])

    def build_model(vocab_size, embedding_dim, rnn_units, batch_size):
    model = tf.keras.Sequential([
        tf.keras.layers.Embedding(vocab_size, embedding_dim,
                                  batch_input_shape=[batch_size, None]),
        tf.keras.layers.LSTM(rnn_units,
                             return_sequences=True,
                             stateful=True,
                             recurrent_initializer='glorot_uniform'),
        tf.keras.layers.Dense(vocab_size)
    ])
    return model

    model = build_model(
      vocab_size=len(vocab),
      embedding_dim=256,
      rnn_units=1024,
      batch_size=1
    )
    
    model.summary()

    Model: "sequential"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    embedding (Embedding)        (1, None, 256)            16640     
    _________________________________________________________________
    lstm (LSTM)                  (1, None, 1024)           525312    
    _________________________________________________________________
    dense (Dense)                (1, None, 86)             88714    
    =================================================================
    Total params: 60,356,426
    Trainable params: 60,356,426
    Non-trainable params: 0
    _________________________________________________________________

    def generate_text(model, start_string):
    num_generate = 1000
    
    input_eval = [char2idx[s] for s in start_string]
    input_eval = tf.expand_dims(input_eval, 0)
    
    text_generated = []
    temperature = 1.0
    
    model.reset_states()
    for i in range(num_generate):
        predictions = model(input_eval)
        predictions = tf.squeeze(predictions, 0)
    
        predictions /= temperature
        predicted_id = tf.random.categorical(predictions, num_samples=1)[-1,0].numpy()
    
        input_eval = tf.expand_dims([predicted_id], 0)
    
        text_generated.append(idx2char[predicted_id])
    
    return (start_string + ''.join(text_generated))
    
    def train_model(model, dataset, epochs, learning_rate, num_examples_to_generate, batch_size=128, buffer_size=10000):
    
    example_buffer = tf.data.Dataset.from_tensor_slices(dataset).shuffle(buffer_size).batch(batch_size)
    
    optimizer = tf.keras.optimizers.Adam(lr=learning_rate)
    
    @tf.function
    def train_step(inp, target):
        with tf.GradientTape() as tape:
            predictions = model(inp)
            loss = tf.reduce_mean(
                tf.keras.losses.sparse_categorical_crossentropy(target, predictions, from_logits=True))
    
        grads = tape.gradient(loss, model.trainable_variables)
        optimizer.apply_gradients(zip(grads, model.trainable_variables))
    
    for epoch in range(epochs):
        start = time.time()
    
        total_loss = 0
    
        for (batch_n, (inp, target)) in enumerate(example_buffer):
            train_step(inp, target)
    
            total_loss += (loss.numpy().mean())
    
        if (epoch + 1) % 1 == 0:
            print ('Epoch {} Loss {:.4f}'.format(epoch+1, total_loss / len(example_buffer)))
            print ('Time taken for 1 epoch {} sec
    '.format(time.time() - start))
    
            generated_text = generate_text(model, start_string=u"ROMEO:")
            print(generated_text)

    epochs = 30
    learning_rate = 0.001
    
    dataset = text_as_int[:-1]
    train_model(model, dataset, epochs, learning_rate, num_examples_to_generate=100, batch_size=128, buffer_size=10000)

    10/10 [==============================] - 21s 205ms/step - loss: 1.6833
    Epoch 2 Loss 1.6833
    Time taken for 1 epoch 19.872640619277954 sec
    
    Epoch 3 Loss 1.6633
    Time taken for 1 epoch 19.89567289352417 sec
    
    Epoch 4 Loss 1.6288
    Time taken for 1 epoch 19.832053470611572 sec
    
    Epoch 5 Loss 1.5688
    Time taken for 1 epoch 19.817887544631958 sec
    
    Epoch 6 Loss 1.4909
    Time taken for 1 epoch 19.774341344833374 sec
    
    Epoch 7 Loss 1.4183
    Time taken for 1 epoch 19.735632181167603 sec
    
    Epoch 8 Loss 1.3698
    Time taken for 1 epoch 19.711107969284058 sec
    
    Epoch 9 Loss 1.3363
    Time taken for 1 epoch 19.711623668670654 sec
    
    Epoch 10 Loss 1.3118
    Time taken for 1 epoch 19.665785741796494 sec
    
    Epoch 11 Loss 1.2887
    Time taken for 1 epoch 19.65223486995697 sec
    
    Epoch 12 Loss 1.2732
    Time taken for 1 epoch 19.7009200339317 sec
    
    Epoch 13 Loss 1.2629
    Time taken for 1 epoch 19.685939025878906 sec
    
    Epoch 14 Loss 1.2513
    Time taken for 1 epoch 19.656405115127563 sec
    
    Epoch 15 Loss 1.2386
    Time taken for 1 epoch 19.660794019708633 sec
    
    Epoch 16 Loss 1.2301
    Time taken for 1 epoch 19.68591680045128 sec
    
    Epoch 17 Loss 1.2255
    Time taken for 1 epoch 19.711800077438354 sec
    
    Epoch 18 Loss 1.2198
    Time taken for 1 epoch 19.73851270198822 sec
    
    Epoch 19 Loss 1.2155
    Time taken for 1 epoch 19.734644203186035 sec
    
    Epoch 20 Loss 1.2087
    Time taken for 1 epoch 19.679280948638916 sec
    
    Epoch 21 Loss 1.2028
    Time taken for 1 epoch 19.68713440990448 sec
    
    Epoch 22 Loss 1.1982
    Time taken for 1 epoch 19.645955701828003 sec
    
    Epoch 23 Loss 1.1943
    Time taken for 1 epoch 19.654324989318848 sec
    
    Epoch 24 Loss 1.1908
    Time taken for 1 epoch 19.72608304977417 sec
    
    Epoch 25 Loss 1.1877
    Time taken for 1 epoch 19.701928329467773 sec
    
    Epoch 26 Loss 1.1848
    Time taken for 1 epoch 19.649063611030578 sec
    
    Epoch 27 Loss 1.1824
    Time taken for 1 epoch 19.71220703125 sec
    
    Epoch 28 Loss 1.1804
    Time taken for 1 epoch 19.71992099761963 sec
    
    Epoch 29 Loss 1.1787
    Time taken for 1 epoch 19.692355394363403 sec
    
    Epoch 30 Loss 1.1772
    Time taken for 1 epoch 19.724089884757996 sec
    
    ROMEO:She turned toward me and said--"Oh, you're very beautiful," he added quickly, his voice beaming with happiness. We kissed each other on the lips before going away together."It's a shame she had to go. I don't think she wants us any longer," she said aloud.I asked how long ago they met. She told me she was coming over tomorrow to Liverpool, where we would see each other again."But what do you mean?" I asked curiously.