Better scoring and map selection + Map delta

AI_TCP_API.md

@@ -22,7 +22,7 @@ any map when no such map exists.
 
 The score is from 0 to 100:
 
-`40 * coin_progress + 40 * distance_progress + 20 * won`
+`50 * coin_progress + 10 * distance_progress + 40 * won`
 
 where `coin_progress` is collected coins divided by the map's total coins,
 `distance_progress` is progress from the spawn point to the finish, and `won`
@@ -37,21 +37,29 @@ are requested separately on every turn.
 ```
 
 Every model gets a separate `Game` instance and can run in parallel. Valid
-requests. Valid moves are `left_down`, `left`, `left_up`, `up`, `right_up`,
+moves are `left_down`, `left`, `left_up`, `up`, `right_up`,
 `right`, and `right_down`. Send `null` for a model when it has no more moves;
 that model is stopped for the current generation.
 
-The response identifies the selected map and returns the initial state:
+The initial response is a `move_request`. It contains the selected map and the
+initial state of every model. The map is sent only in this initial response;
+clients should cache it for the rest of the simulation.
 
 ```json
 {
   "ok":true,
   "type":"move_request",
   "map":{"name":"default","difficulty":30.0,"layout":[["stone", "stone"]]},
-  "results":{"model-a":{"won":false,"coins":0,"moves":0,"position":[1,4],"finished":false},"model-b":{"won":false,"coins":0,"moves":0,"position":[1,4],"finished":false}}
+  "results":{"model-a":{"won":false,"coins":0,"moves":0,"position":[1,4],"finished":false,"changes":[]},"model-b":{"won":false,"coins":0,"moves":0,"position":[1,4],"finished":false,"changes":[]}}
 }
 ```
 
+After receiving this response, send one move for every model:
+
+```json
+{"type":"step","moves":{"model-a":"right","model-b":"right_up"}}
+```
+
 ## Advance one turn
 
 The client must send exactly one move for every model on each turn. The server
@@ -62,10 +70,49 @@ predict the next moves.
 {"type":"step","moves":{"model-a":"right","model-b":null}}
 ```
 
-The response is `move_request` with the selected map and updated per-model states, asking the
+The response is another `move_request` containing the updated state and score
-client for the next move of every model. When every model has finished, the
+for every model. Each model also receives `changes`, a cumulative list of
-response type becomes `simulation_result` and includes the best model's
+blocks that differ from the base map. For example, a collected coin is sent as
-server-computed `score`, which is used to choose the next training map.
+`{"position":[4,7],"type":null}`. Unchanged turns return an empty list. It
+asks the client for the next move of every unfinished model. A model that has
+no more moves can send `null`:
 
-The result score is `1000` for a win, plus `10` per coin, minus the number of
+```json
-moves. Malformed requests receive `{"ok":false,"error":"..."}`.
+{"type":"step","moves":{"model-a":"right","model-b":null}}
+```
+
+When every model has finished, the response type becomes `simulation_result`.
+It contains the final state and score for every model, plus the best score used
+to choose the next training map:
+
+```json
+{
+  "ok":true,
+  "type":"simulation_result",
+  "results":{
+    "model-a":{
+      "won":true,
+      "coins":3,
+      "moves":18,
+      "position":[23,1],
+      "finished":true,
+      "changes":[{"position":[4,7],"type":null}],
+      "score":92
+    },
+    "model-b":{
+      "won":false,
+      "coins":1,
+      "moves":7,
+      "position":[8,4],
+      "finished":true,
+      "changes":[],
+      "score":24
+    }
+  },
+  "score":92
+}
+```
+
+The score is calculated by the server using the weights described above. The
+top-level `score` is the best model score for the completed generation.
+Malformed requests receive `{"ok":false,"type":"error","error":"..."}`.

README.md

@@ -0,0 +1,4 @@
+# Améliorations
+
+- Pénaliser les mouvements impossibles : intéressant, même pour l'algo génétique, pourrait optimiser le genome
+- Pénaliser le fait de terminer la simulation sur une case où l'on est déjà passé ou pénaliser de revenir sur une case où l'on est passé

game/handlers/aiHttpGameHandler.py

@@ -24,6 +24,7 @@ of moves to be executed in a single game simulation, based on the map min_moves
 """
 
 import json
+import math
 import os
 import random
 import socketserver
@@ -43,12 +44,13 @@ from game.mapSet import MapSet
 class AiHttpGameHandler(GameHandler):
     DEFAULT_HOST = "127.0.0.1"
     DEFAULT_PORT = 8765
-    VALIDATION_DIFFICULTY = 70
+    VALIDATION_DIFFICULTY = 90
     REPLAY_FRAME_DELAY = 0.1
     # Scoring weights TOTAL MUST BE 100
-    COINS_WEIGHT = 40
+    COINS_WEIGHT = 50
-    DISTANCE_WEIGHT = 20
+    STEPS_WEIGHT = 10
-    WIN_WEIGHT = 40
+    DISTANCE_WEIGHT = 10
+    WIN_WEIGHT = 30
 
     def __init__(self, host: str | None = None, port: int | None = None,
                  max_workers: int | None = None):
@@ -62,6 +64,10 @@ class AiHttpGameHandler(GameHandler):
             game_map for game_map in self.map_set.maps
             if game_map.difficulty >= self.VALIDATION_DIFFICULTY
         ] or list(self.map_set.maps)
+        
+        if (self.COINS_WEIGHT + self.DISTANCE_WEIGHT + self.STEPS_WEIGHT + self.WIN_WEIGHT) != 100:
+            raise ValueError("Scoring weights must sum to 100.")
+                
     def run(self) -> None:
         handler = self
 
@@ -73,6 +79,7 @@ class AiHttpGameHandler(GameHandler):
                     "best_run": None,
                     "replay_condition": Condition(),
                     "client_connected": True,
+                    "stagnant_generations": 0,
                 }
                 replay_thread = Thread(target=handler._replay_best_runs, args=(session,))
                 replay_thread.start()
@@ -81,12 +88,9 @@ class AiHttpGameHandler(GameHandler):
                     for raw_request in self.rfile:
                         try:
                             request = json.loads(raw_request)
-                            # print(f"AI request received: {request.get('type')} : {request}", flush=True)
                             response = handler._handle_request(request, session)
                         except (json.JSONDecodeError, TypeError, ValueError, KeyError) as error:
                             response = {"ok": False, "type": "error", "error": str(error)}
-                            # print(f"AI request error: {error}", flush=True)
-                        # print(f"AI response sent: {response.get('type')}", flush=True)
                         compact_response = json.dumps(response, separators=(",", ":"))
                         self.wfile.write((compact_response + "\n").encode("utf-8"))
                         self.wfile.flush()
@@ -189,7 +193,7 @@ class AiHttpGameHandler(GameHandler):
             simulation = simulations[model_id]
             # print(f"Starting step for model '{model_id}'", flush=True)
             result = simulation.step(move)
-            result["score"] = self._score_result(simulation, result)
+            result["score"] = self._score_simulation(simulation, result)
             # print(f"Finished step for model '{model_id}'", flush=True)
             return model_id, result
 
@@ -198,12 +202,33 @@ class AiHttpGameHandler(GameHandler):
         # print(f"Completed step for models: {list(moves)}", flush=True)
         all_finished = all(result["finished"] for result in results.values())
         response = {"ok": True, "type": "move_request", "results": results}
+        
         if all_finished:
             response["type"] = "simulation_result"
             best_score = max(result["score"] for result in results.values())
             best_model_id = max(results, key=lambda model_id: results[model_id]["score"])
+            map_difficulty = session["map"].difficulty  # e.g., 20 or 60
             response["score"] = best_score
-            session["performance"] = best_score
+
+            # --- Curriculum Regression (Fallback) & Performance Update ---
+            PASSING_THRESHOLD = 85.0  # Score percentage considered successful completion
+            if best_score < PASSING_THRESHOLD:
+                session["stagnant_generations"] = session.get("stagnant_generations", 0) + 1
+            else:
+                session["stagnant_generations"] = 0  # Reset on breakthrough
+
+            # If stuck on a difficulty wall for 15 generations, fallback to an easier map range
+            if session["stagnant_generations"] >= 15:
+                current_perf = session.get("performance", 10 * math.sqrt(map_difficulty))
+                regression_drop = 15.0
+                session["performance"] = max(0.0, current_perf - regression_drop)
+                print(f"Stuck on difficulty {map_difficulty} for {session['stagnant_generations']} generations. Dropping target performance to {session['performance']:.1f}", flush=True)
+                session["stagnant_generations"] = 0  # Reset stagnation counter after fallback
+            else:
+                difficulty_scaling = 10 * math.sqrt(map_difficulty)
+                normalized_score = best_score / 100.0  # Convert 0-100 percentage to 0.0-1.0 scale
+                session["performance"] = difficulty_scaling * (0.5 + 0.5 * normalized_score)
+
             session["last_result"] = response
             session["simulations"] = None
             session["best_run"] = (
@@ -252,6 +277,8 @@ class AiHttpGameHandler(GameHandler):
                     line += "\tMoves: " + str(game.gameState.movesCount)
                 elif index == 2:
                     line += "\tGeneration: " + str(generation)
+                elif index == 3:
+                    line += "\tMove: " + moves[game.gameState.movesCount - 1] if len(moves) >= game.gameState.movesCount else "\tMove: None"
                 frame_lines.append(line)
             print("\033[2J\033[H\n\n\n" + "\n".join(frame_lines), flush=True)
             sleep(self.REPLAY_FRAME_DELAY)
@@ -264,20 +291,34 @@ class AiHttpGameHandler(GameHandler):
         results = {}
         for model_id, simulation in simulations.items():
             result = simulation.result()
-            result["score"] = self._score_result(simulation, result)
+            result["score"] = self._score_simulation(simulation, result)
             results[model_id] = result
         return results
 
     def _select_map(self, performance: float, validation: bool):
         if validation:
             return random.choice(self.validation_maps)
-        return self.map_set.get_maps_by_difficulty(round(performance), 1)[0]
+        maps = self.map_set.get_maps_by_difficulty(round(performance), 2)
+        return random.choice(maps)
 
-    def _score_result(self, simulation: AiSimulation, result: dict) -> int:
+    def _score_simulation(self, simulation: AiSimulation, result: dict) -> int:
-        if (self.COINS_WEIGHT + self.DISTANCE_WEIGHT + self.WIN_WEIGHT) != 100:
-            raise ValueError("Scoring weights must sum to 100.")
-        
         game_map = simulation.game.map
+        raw_score = self._score_result(game_map, result)
+        max_score = self._get_map_max_score(game_map)
+        if max_score == 0:
+            return 0
+        percentage = (raw_score / max_score) * 100.0
+        return round(max(0.0, min(100.0, percentage)))
+    
+    def _get_map_max_score(self, game_map: Map) -> int:
+        result = {}
+        result["position"] = game_map.get_first_block_type_position("finish")
+        result["coins"] = game_map.count_blocks_of_type("coin")
+        result["won"] = True
+        result["moves"] = game_map.min_moves
+        return self._score_result(game_map, result)
+        
+    def _score_result(self, game_map: Map, result: dict) -> int:
         finish = game_map.get_first_block_type_position("finish")
         spawn = game_map.get_first_block_type_position("spawn_point")
         total_coins = game_map.count_blocks_of_type("coin")
@@ -290,11 +331,13 @@ class AiHttpGameHandler(GameHandler):
             distance_progress = 1.0 if start_distance == 0 else max(
                 0.0, min(1.0, 1.0 - remaining_distance / start_distance)
             )
+            steps_progress = 0.0 if result["moves"] == 0 else min(1.0, 1.0 - result["moves"] / game_map.min_moves)
 
         coin_progress = 0.0 if total_coins == 0 else min(1.0, result["coins"] / total_coins)
         return round(
             self.COINS_WEIGHT * coin_progress
             + self.DISTANCE_WEIGHT * distance_progress
+            + self.STEPS_WEIGHT * steps_progress
             + self.WIN_WEIGHT * result["won"]
         )
 

game/handlers/aiSimulation.py

@@ -14,6 +14,7 @@ class AiSimulation:
 
     def __init__(self, game_map: Map):
         self.game = Game(deepcopy(game_map), GameState())
+        self._base_map_types = self.game.map.to_type_dict()
         min_moves = self.game.map.get_min_moves()
         self.max_moves = min_moves * 2 if min_moves is not None else None
         self._condition = Condition()
@@ -65,8 +66,18 @@ class AiSimulation:
             "moves": state.movesCount,
             "position": list(state.get_player_position()),
             "finished": state.stopGame,
+            "changes": self._get_map_changes(),
         }
 
+    def _get_map_changes(self) -> list[dict]:
+        current_map_types = self.game.map.to_type_dict()
+        changes = []
+        for y, row in enumerate(current_map_types):
+            for x, block_type in enumerate(row):
+                if block_type != self._base_map_types[y][x]:
+                    changes.append({"position": [x, y], "type": block_type})
+        return changes
+
     def close(self) -> None:
         self.stop_for_generation()
 

game/map.py

@@ -71,6 +71,12 @@ class Map:
     def to_type_dict(self):
         """Returns a matrix containing block types and None for empty spaces."""
         return [
-            [block.type if block is not None else None for block in row]
+            [self._visible_block_type(block) for block in row]
             for row in self.layout
-        ]
+        ]
+
+    @staticmethod
+    def _visible_block_type(block):
+        if block is None or getattr(block, "collected", False):
+            return None
+        return block.type

maps/big_peanut.toml

@@ -18,7 +18,7 @@ layout = """
    ##     °  ###^########        
     ##    # ##########           
      #S  ##^#####                
-      #########                  
+     ##########                  
 """
 
 # Ya moyen d'optimiser le trajet sur les surfaces plates en faisant des sauts au lieu de marcher normalement

maps/big_peanut_pirate.toml

@@ -0,0 +1,24 @@
+difficulty = 100
+min_moves = 40
+layout = """
+                #####            
+             ###### ########     
+            ##             ####  
+          ###               °E## 
+        ###              ° °#####
+      ##°°°   ° °° ° ° °## ######
+      #°####### ## # # ### ######
+     # #°    °°°              ## 
+    ## ##°° ° ##°°        ^###   
+    #   ###°° ####°       ##     
+    #   #####^#####°°    ##      
+   #       #####°####°   #       
+   #           ° °°°°#   #       
+   #       ° °°# #####^^##       
+   ##     ° °###^########        
+    ##   °# ##########           
+     #S °##^#####                
+     ##########                  
+"""
+
+# Big_peanut avec des pièces partout (HAARRRRRR !)

maps/default.toml

@@ -14,6 +14,56 @@ layout = """
 
 
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 #000000000000000000000000000000KKKKKKKKKKKKKKKKKKXXKKKKKXXXXNWMMNNXXXNNNNNNNNWWWWWMMMMMMMMMMMMMMMMMMM
 #0000000000000000000000000000000KKKKKKKKKKKKKKKKKXWWXKKKKXXXNNWNXXXXXXNNNNNNNMWWWMMMMMMMMMMMMMMMMMMMM
 #00000000000000000000000000000000KKKKKKKKKKKKKKKKXMMNKKKKXXXXXXXXXXXXXXNNNNNNNWWWWMMMMMMMMMMMMMMMMMM0

maps/distance_tutorial.toml

@@ -1,7 +1,7 @@
-difficulty = 20
+difficulty = 30
 min_moves = 28
 layout = """
-#                           #
+#   ° ° ° ° ° ° ° ° ° ° °   #
 #E                          #
 ##########################°^#
 #     #°#°#  #°#°#  #°#°#°###

maps/fall_tutorial.toml

@@ -1,4 +1,4 @@
-difficulty = 10
+difficulty = 25
 min_moves = 12
 layout = """
 ########            
@@ -8,7 +8,7 @@ layout = """
   #          ##     
   #           ###   
   #             ##  
-  #  ##          ## 
+  # °##          ## 
   #  ###          ##
   #   °            #
   #   ^            #

maps/home.toml

@@ -1,4 +1,4 @@
-difficulty = 13
+difficulty = 20
 min_moves = 24
 layout = """
                                     E##

maps/jump_tutorial.toml

@@ -1,4 +1,4 @@
-difficulty = 15
+difficulty = 10
 min_moves = 14
 layout = """
 

maps/lava_bucket.toml

@@ -1,4 +1,4 @@
-difficulty = 60
+difficulty = 70
 min_moves = 6
 layout = """
       #             S             #

maps/straight_line.toml

@@ -1,4 +1,4 @@
-difficulty = 0
+difficulty = 5
 min_moves = 11
 layout = """
 #                       #

maps/straight_line_inversed.toml

@@ -0,0 +1,9 @@
+difficulty = 7
+min_moves = 11
+layout = """
+#                       #
+#E                     S#
+#########################
+"""
+
+# Teach to not only go right

maps/the_wall_is_lava.toml

@@ -1,18 +1,18 @@
 difficulty = 40
-min_moves = 6
+min_moves = 8
 layout = """
 #^  ^#
 #^ S^#
 #^ #^#
 #^  ^#
 #^# ^#
-#^  ^#
+#^ °^#
 #^ #^#
 #^# ^#
 #^ #^#
 #^# ^#
 #^  ^#
-#^  ^#
+#^ °^#
 #^ #^#
 #^  ^#
 #^  ^#