Updating one tables column values based on values in another table

I am displaying a series of boxes in a grid, with a background color set by a value in a table called objectives… There is a Color table that has a hex color that corresponds to the value. I use that since changing the color there will change it globally in the program. I use a column sled value instead of just id to make it easier to change a reference to the color and not have to track deletion of rows.

Once the values are calculated, I lookup and update the color in Objectives with the corresponding color in Color.

I have two questions;

  1. While this works, it seems excessive. Is there a better way to update the colors based on changed values than:
        objective_num=Objectives.objects.filter(dashboard=dashboard_id).values('id','objective_color')#get dic of latest color values for each id
        for n in objective_num: #Iterate over all the objectives, starting at 0 to valeu of objective_num
            color_hex = n.get('objective_color') #get color code to use to get coressponding hex value from Color table
            obj_id=n.get('id') #get objective id for the color
            color = Color.objects.filter(value=color_hex).values_list('objective_background', flat=True) #get color hex value that natches teh color set in Objectives
            Objectives.objects.filter(dashboard=dashboard_id).filter(id=obj_id).update(objective_background=color) #update color hex in Objectives table

  1. While it is intended only to use current values, someone will want historical ones at some point. I was thinking about creating a historical table, where I add a column every update, a very Excel way of doing it. Alternatively, I could add rows with each update and number them to be able to get historical data. That way, I need only save the change values. Any suggestions as to which approach is better?

Models

class Color(models.Model):
    color=models.CharField(max_length=10, default = 'White')
    objective_background = models.CharField(max_length=7, default='#FFFFFF')
    value=models.IntegerField( default = 1)

def __str__(self):
        return f"{self.color} (Hex: {self.objective_background }) is color number {self.value} with ID {self.id}"
    
class Objectives(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    dashboard=models.ForeignKey(Dashboard, on_delete=models.CASCADE)
    objective_background = models.CharField(max_length=8, default=1)
    row = models.ForeignKey(Row, on_delete=models.CASCADE)
    objective_position = models.IntegerField()
    objective_color = models.IntegerField() #Used to store status color based on performance
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)

def __str__(self):
        return f" Objective: {self.row} {self.objective_position} {self.objective_text} at {self.timestamp}"
   

You want to start thinking in terms of the models as objects. Work with the objects and the native collections (querysets). You really don’t want to use the values (or values_list) function unless you need to.

I’m not sure I’m entirely following your intent here, but this seems to boil down to:

for n in Objectives.objects.filter(dashboard=dashboard_id):
    n.objective_background = Color.objects.get(value=n.objective_color).objective_background
    n.save()

Which, if this is a true expression of the requirement, implies to me that your objective_color field should really be a foreign key to Color, removing the need for the redundant reference.

(In your model, these definitions are redundant and potentially conflicting:)

objective_background = models.CharField(max_length=8, default=1)
objective_color = models.IntegerField()

Definitely by adding rows. Or possibly by copying the old entry to a History model for tracking changes. (Choice may depend upon the frequency of changes.)

You would never want to add a column (field) to a model based on updating data.

What I am doing is:

  1. A score is calculated for the objective and a color is assigned to based on the score.
  2. The Hex Value used to change the background color in the css is then retrieved from the color table and put into the objectives table.
  3. I then iterate over the objectives table in my template to render the page

I tried using an FK to Color in Objective initially, but when I tried to change it in the Objectives table I got a key error. It didn’t like me replacing the existing key with a new one; even though both key values existed. I’m not sure what went wrong when I tried to update the FK value in Objectives.

I wound up doing that because I could get the value in objective_color and use it to find the corresponding hex value in the Color table and then store it in objective_background.

It also made it easier to count the number of objectives of each color without having to remember the hex values, which would change if the client wants a different shade while objective_color stays the same no matter the hex value.

It’s do-able. We would need to see the code to figure out what was wrong.

It really doesn’t make it easier. Again, showing the code being used would help with the diagnosis.

1 Like

I rewrote it using an FK (I didn’t save the earlier nonworking version) and whatever I messed up before I fixed in my new code:

        color = 1 #Test value to see if changing FK works
        for n in Objectives.objects.filter(dashboard=dashboard_id):
            Objectives.objects.update(hex=color)
            n.objective_background = Color.objects.get(value=color).objective_background
            n.save()

When I write score calculation function (I’m taking it one working step at a time), I can delete the extra objects (right term?) in the model and just change the FK at the same time, rather than pass the scores to another function and update there. That would eliminate the line of code:

Objectives.objects.update(hex=color)

since it would already be updated. I could save the old values inside a history table before updating.

Interestingly, if tried to update then get color in two loops I get:

Field 'value' expected a number but got <Color: Yellow (Hex: #F7DC6F) is color number 3 with ID 3>.

where it is using the str def

Here’s the code I tried:

        color = 1 #Test value to see if changing FK works
        for n in Objectives.objects.filter(dashboard=dashboard_id):
            Objectives.objects.update(hex=color)
        for n in Objectives.objects.filter(dashboard=dashboard_id):
            n.objective_background = Color.objects.get(value=n.hex).objective_background
            n.save()

Thanks again for the suggestion.

Model

class Objectives(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    dashboard=models.ForeignKey(Dashboard, on_delete=models.CASCADE)
    row = models.ForeignKey(Row, on_delete=models.CASCADE)
    hex = models.ForeignKey(Color, on_delete=models.CASCADE, default=1) #added to test FK
    objective_background = models.CharField(max_length=8, default=1) # (not needed)
    objective_position = models.IntegerField()
    objective_color = models.IntegerField() #Used to store status color based on performance. 
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)

Template:

 {% for objtext in rows %}  
 <tr><td>

  <!--Create row of topic shapes and spacer between shapes and apply status color-->
  <!--Create pointed arrow and space to Outcome-->

         {% for a_text in objtext %}

            {% if a_text.objective_position == 1 %}
              <div class="item_LoO_Name">
                {{ a_text.objective_text|linebreaks}}</div>
            {% else %}
              {% if a_text.objective_position != 10 %}
                  <div class="space"></div>
                 <div class="Objective" style= 'background-color:{{ a_text.objective_background}}'>
                        {{ a_text.objective_text|linebreaks}}</div> 
              {% else %}
                <div class="space"></div>
                <div class="triangle-right"></div>
                <div class="spaceblank"></div>
                <div class="item_Outcome_Name">
                    {{ a_text.objective_text|linebreaks}}</div>
              {% endif %} 
            {% endif %}
                  {% endfor %}

          {% endfor %}

When trying to set a foreign key to the id and not the object, you need to use the _id identifier on the field. i.e.:
Objectives.objects.update(hex_id=color)
See the docs for Field lookups

Otherwise, the code you would want would assign the object as the reference, and so would look like:

color = Color.objects.get(id=1)
for n in Objectives.objects.filter(dashboard=dashboard_id):
    Objectives.objects.update(hex=color)

If I understand the docs correctly, using hex_id FK in Objectives references the PK of the model color, so I could use:

    for n in Objectives.objects.filter(dashboard=dashboard_id):
        Objectives.objects.update(hex_id=color)
        n.objective_background = Color.objects.get(pk=hex_id).objective_background
        n.save()

Which makes the column ‘value’ redundant (since it just duplicates the hex_id anyway, and instead of changing the value attribute if a user wants to change what color is say, #2, I would just change the hex value in the table Color instead of changing the value.

What’s the purpose or intent of this line?

What it’s actually going to do is change all of the Objectives to set hex_id=color every time through the loop - undoing what you’re doing in the next two lines.

Yes, this (below) is a bad edit:

For what you have, wanting to set all objectives with dashboard_id=dashboard_id, it would more appropriately be:

Also true.

I see what you are saying, and made some changes in the code so it works properly, though a bit inelegant, IMHO:

       **# Simulate having list of colors based on calculated score - will be done later based on actual data as separate function**

        objid =  1 # set counter of Objective 1d to 1
        for m in Objectives.objects.filter(dashboard=dashboard_id):
            color = random.randint(1,4) #will be replaced with code to fetch actual score for objective n to insert into color
            objid= m.id
            Objectives.objects.filter(id=objid).update(hex_id=color) #insert color based on score

        **# Update objective background hex based on color value - actual code to be used**

        for n in Objectives.objects.filter(dashboard=dashboard_id):
            objid= n.id #get objective id number, used n.id since each dashbaord will have unique set, not 1-60 like first
            color = list(Objectives.objects.filter(id=objid).values_list('hex_id', flat=True)) #get color for objective id
            objcolor=color[0] #convert to integr by getting value from list
            n.objective_background = Color.objects.get(pk=objcolor).objective_background #get color based on objcolor value 1 - x
            Objectives.objects.filter(id=objid).update(objective_background=n.objective_background) #update hex value in table

The logic behind this is:

  1. Data is collected on each objective and analyzed to result in a score
  2. The score is then assigned a color, usually red, green or yellow
  3. That score color value is inserted into the Objectives table for each objective, in a standalone function
  4. In the createdashboard function, the color value is fetched for each objective and the corresponding hex color inserted into the table and used by the template to display the appropriate color

While it is a bit duplicative to store color and hex_id, if I have color if at some point a client decides to change the color or shade if they recall historical data it will be changed automatically if the data is used by a template. I would not have to recalculate all the data values but just fetch the historical color value and use the new hex value.

The code works as desired but I appreciate any suggestion for improvements.

You’re still not thinking of this in terms of the objects involved, and doing a lot of “manual” work.

Commentary on the existing code:

This simplifies to:

Similarly, if you correct your model such that the reference to the Color model becomes a foreign key, the entire need for second loop goes away.

This would also be possible if color is a foreign key. The only work necessary would be to change the fk reference from the current row to the historical row.

[quote=“KenWhitesell, post:11, topic:36084”]

Thanks, I just forgot to do that when I rewrote the code. I usually comment it out to remember to remove it from the final version if I don’t delete it right away.

I must have done something wrong because whenever I tried to use m.id I got errors of the type expected x but got y.

I did another rewrite to:


        for m in Objectives.objects.filter(dashboard=dashboard_id):
            m.hex_id = random.randint(1,4)
            m.save()
        # Update objective background hex based on color value
        for n in Objectives.objects.filter(dashboard=dashboard_id):
            n.objective_background = Color.objects.get(pk=n.hex_id).objective_background #get color based on objcolor value 1 - x
            n.save()

and it works.

If I understand what you are saying, once a value has been assigned to an object (that creates an instance) that value is fixed until change and can be returned by using the object, there is no need to create another variable to use the value.

hex_id is actually an FK to ‘Color’, I put the actual hex value into the table because that was the only way I could figure out how to pass it in context the the template to set the new background color; while I try to see how to get something like

{{ a_text.objectives.get.hex_id }} 

to replace

{{ a_text.objective_background}}

in:

        {% for a_text in objtext %}

            {% if a_text.objective_position == 1 %}
              <div class="item_LoO_Name">
                {{ a_text.objective_text|linebreaks}}</div>
            {% else %}
              {% if a_text.objective_position != 10 %}
                  <div class="space"></div>
                  <a href="{% url 'testpage' %}">
                 <div class="Objective" style= 'background-color:{{ a_text.objective_background}}'>
                        {{ a_text.objective_text|linebreaks}}</div>
                 </a>
              {% else %}

Once I do then I won’t need to store the hex value, instead teh template can fetch teh coressponding hex value based on hex_id as a FK.

Model


class Objectives(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    dashboard=models.ForeignKey(Dashboard, on_delete=models.CASCADE)
    row = models.ForeignKey(Row, on_delete=models.CASCADE)
    hex = models.ForeignKey(Color, on_delete=models.CASCADE, default=1) #added to test FK
    objective_background = models.CharField(max_length=8, default=1)
    objective_position = models.IntegerField()
    objective_color = models.IntegerField() #Used to store status color based on performance Can be deleted as not needed
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)


    def __str__(self):
        return f" Objective: {self.id} is in row {self.row} {self.objective_position} {self.objective_text} at {self.timestamp}"

That is correct.

Assuming:

  • a_text is an instance of ObjectiveText
  • The field objectives in ObjectiveText is an FK to Objectives
  • The model Objectives has a ForeignKey field named color referencing the model Color
  • The model Color as a field named hex to be used in the template

Then:

  • If color is an instance of Color, then color.hex is the hex field in that instance of Color
  • If objectives is an instance of Objectives, then objectives.color is an instance of Color.
    • and, by extension of the first point then objectives.color.hex is the hex field of Color
  • If a_text is an instance of ObjectiveText, then a_text.objectives is an instance of Objectives

Putting this all together means then that a_text.objectives.color.hex is the reference to the hex field of the Color object that is related to a_text through the chain of foreign keys.

I deleted ObjectiveText and just moved all the text into a field in objectives since all there was in ObjectiveText was text.

In Objectives, hex is the FK to Color and the label is hex_id since it is a FK.
Color has a field, objective_background, that contains the hex value to use as background.

class Color(models.Model):
    color=models.CharField(max_length=10, default = 'White')
    objective_background = models.CharField(max_length=7, default='#FFFFFF')
    value=models.IntegerField( default = 1) #redundant, to be deleted
 

From your explanation and what I could find, you can use a PK to get a field value via the FK to the model.

Based on that, I tried, for example:

<div class="Objective" style= 'background-color:{{ a_text.objectives.hex_id.background_color }}'>

and
<div class="Objective" style= 'background-color:{{ a_text.objectives.hex.background_color }}'>

Since hex is the FK in Objectives to Color and background_color the desired value to insert in the HTML.

I tried a number of permutations but the HTML was always blank

                 <div class="Objective" style= 'background-color:'>
                        <p></p></div>

Here is the code in views (I also tried hex_id.):

def createdashboard(dashboard_id):
       # Simulate having list of colors based on calaculated score - will be done later based on actual data as seperate function
        for m in Objectives.objects.filter(dashboard=dashboard_id):
            m.hex_id = random.randint(1,4)
            m.save()
        # Update objective background hex based on color value
        for n in Objectives.objects.filter(dashboard=dashboard_id):
            n.objective_background = Color.objects.get(pk=n.hex_id).objective_background #get color in hex, use hex_id as fk to Color to select proper hex value
            n.save()


        rows = [] # create blank dic to collect dictionary of row data
        for i in range(1,7): # create 6 rows of data
            data2=(Objectives.objects.filter(
                                dashboard = dashboard_id, row=i
                                ).order_by('id').values("objective_text", 'objective_position', 'objective_background', 'hex') # Text is for display, position used to id if Project or Outcome
            )
            rows.append(data2)
        return (data2, rows)

Template section (I also tried hex_id.):

{% for objtext in rows %}  
 <tr><td>

  <!--Create row of topic shapes and spacer between shapes and apply status color-->
  <!--Create pointed arrow and space to Outcome-->

         {% for a_text in objtext %}

            {% if a_text.objective_position == 1 %}
              <div class="item_LoO_Name">
                {{ a_text.objective_text|linebreaks}}</div>
            {% else %}
              {% if a_text.objective_position != 10 %}
                  <div class="space"></div>
                  <a href="{% url 'testpage' %}">
                 <div class="Objective" style= 'background-color:{{ a_text.objectives.hex_id.background_color }}'>
                        {{ a_text.objective_text|linebreaks}}</div>
                 </a>

It seems like you’ve made a number of structural changes. Please post the current versions of the other models as well.

Sorry, been making changes as I learn more. Here are all the models:

class Color(models.Model):
    color=models.CharField(max_length=10, default = 'White')
    objective_hex = models.CharField(max_length=7, default='#FFFFFF')
 
    def __str__(self):
        return f"{self.color} (Hex: {self.objective_background }) is color number {self.value} with ID {self.id}"
    
class Dashboard(models.Model):
    dashboard = models.CharField(max_length=50)
    company = models.CharField(max_length=150, default="None")
   

    def __str__(self):
        return f"Company: {self.company} Dashboard: {self.dashboard}"

class Row(models.Model):
    row = models.IntegerField()
    rowhex=models.ForeignKey(Color, on_delete=models.CASCADE, default=1)

    def __str__(self):
        return f"Row; {self.row} with ID {self.id}"

class Objectives(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    dashboard=models.ForeignKey(Dashboard, on_delete=models.CASCADE)
    row = models.ForeignKey(Row, on_delete=models.CASCADE)
    hex = models.ForeignKey(Color, on_delete=models.CASCADE, default=1) #added to test FK
    objective_background = models.CharField(max_length=8, default=1)
    objective_position = models.IntegerField()
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)


    def __str__(self):
        return f" Objective: {self.id} is in row {self.row} {self.objective_position} {self.objective_text} at {self.timestamp}"
    
class UserProfile(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    company=models.CharField(max_length=250, null = True)

    def __str__(self):
        return f'Username: {self.user}  Name: {self.user.last_name}  Company:{self.company}'
    

    

That’s ok - but that is why it’s helpful to post the current code as needed.

So, if you have an instance of Objectives named objective, the hex code would be referenced as

objective.hex.objective_hex
              ^^^^^^^^^^^^^ The color code
          ^^^ Instance of Color
^^^^^^^^ Instance of Objectives

Or, what may be the case in your template, it looks like a_text is the Objectives instance, in which case it would be: a_text.hex.objective_hex