Values updating automatically inside the for loop

Shifat E Arman

I was trying to implement the value iteration algorithm. I have a grid

grid = [[0, 0, 0, +1],
    [0, "W", 0, -1],
    [0, 0, 0, 0]]

An actionlist

actlist = {UP:1, DOWN:2, LEFT:3, RIGHT:4}

And a reward function

reward = [[0, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 0,  0]]

I wrote a function T, which returns tuple of 3 tuples.

def T(i,j,actions):
if(i == 0 and j == 0):
    if(actions == UP):
        return (i,i,0.8),(i,i,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,i,0.1),(i+1,j,0.1)
elif (i == 0 and j == 1):
    if(actions == UP):
        return (i,i,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i,j,0.1)
elif(i == 0 and j == 2):
    if(actions == UP):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return(i+1,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i+1,j,0.1)
elif(i == 0 and j == 3):
    if(actions == UP):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == DOWN):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == LEFT):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == RIGHT):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
# 2nd row
elif (i == 1 and j == 0):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
elif(i == 1 and j ==1):
    if(actions == UP):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
elif (i == 1 and j == 2):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i+1,j,0.1)
elif(i == 1 and j == 3):
    if(actions == UP):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == DOWN):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == LEFT):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == RIGHT):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)      
# 3rd row
elif(i == 2 and j == 0):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j,0.1),(i,j+1,1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i,j,0.1)
elif (i == 2 and j == 1):
    if(actions == UP):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i,j,0.1)
elif(i == 2 and j == 2):
    if(actions == UP):
        return (i-1,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i-1,j,0.1),(i,j,1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i,j,0.1)
elif(i == 2 and j == 3):
    if(actions == UP):
        return (i-1,j,0.8),(i,j-1,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i-1,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i-1,j,0.1),(i,j,0.1)

This function is called in the value iteration function:

def value_iteration():
U1 = [[0, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 0, 0]] 
while True:
    U=U1.copy()
    delta = 0
    for i in range(len(grid)):
        for j in range(len(grid[i])):
            U1[i][j] = max(sum(p*(R(k,l)+gamma*U[k][l]) for (k,l,p) in T(i,j,a)) for a in actlist)
            print(i,j,U1[i][j],U[i][j])
            delta = max(delta, abs(U1[i][j] - U[i][j]))
    if delta <= epsilon*(1 - gamma)/gamma:
        return U

I was updating

U=U1.copy()

in the while loop.

The problem is, the output looks like this:

0 0 0.0 0.0
0 1 0.0 0.0
0 2 0.0 0.0
0 3 1.0 1.0
1 0 0.0 0.0
1 2 0.0 0.0
1 3 -1.0 -1.0
2 0 0.0 0.0
2 1 0.0 0.0
2 2 0.7000000000000001 0.7000000000000001
2 3 0.9630000000000001 0.9630000000000001

But I was not updating U inside the for loops. U was supposed to remain unchanged (that means, all zeros) and U1 was only supposed to change. Why U become automatically set to the value of U1 inside the for loop?

nimish

U1 (and U) is a list of lists, really a list of references to lists.

You're (shallow) copying the outer list, but the contents of the copy are still references to the same inner lists.

Try:

import copy
U = copy.deepcopy(U1)

and see what happens instead. deepcopy will correctly recursively copy the lists.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at2020-11-30

Comments

0 comments

Values updating automatically inside the for loop

Values updating automatically inside the for loop

pump.io port in URL

How to import an asset in swift using Bundle.main.path() in a react-native native module

Failed to listen on localhost:8000 (reason: Cannot assign requested address)

Inner Loop design for webscrapping

Can't pre-populate phone number and message body in SMS link on iPhones when SMS app is not running in the background

mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on '127.0.0.1:3306' (111 Connection refused)

Removed zsh, but forgot to change shell back to bash, and now Ubuntu crashes (wsl)

ggplotly no applicable method for 'plotly_build' applied to an object of class "NULL" if statements

How to run blender on webserver?

Resetting Value of <input type="time"> in Firefox

Converting a class method to a property with a backing field

Ambiguous use of 'init' with CFStringTransform and Swift 3

Execute ./script.sh with a crontab

How to set tab order for array of cluster,where cluster elements have different data types in LabVIEW?

How to pass data to the ng2-bs3-modal?

Retrieve Element Tag Value XML Using Bash

Spring Boot JPA PostgreSQL Web App - Internal Authentication Error

SQL Server : need add a dot before two last character

Making Array From Page Elements in jQuery

Laravel's ORM sync with timestamps doesn't update timestamps

Do animations stop css changes after animation completion?